In my previous blog post I discussed a bit about software defined options using Hyper-V https://msandbu.wordpress.com/2015/07/28/software-defined-storage-options-for-hyper-v/ and that Windows Server is getting alot of good built-in capabilities but lacks the proper scale out solution with performance, which is also something that is coming with Windows Server 2016.
Now one of the vendors which I talked about which has a proper scale-out SDS solution for Hyper-V with support for SMB 3 is Nutanix, which is the subject for this blogpost where I will describe how it works for SMB based storage, now before I head on over to that I want to talk a little bit about how SMB 3 and some of the native capabilities and why they do not work for a proper HCI setup.
With SMB 3.0 Microsoft Introduced two great new features, which was SMB Direct and Multichannel, which are features that are aimed for higher troughput over lower latency.
SMB Multichannel (leverages multiple TCP connections across multiple CPU cores using RSS)
SMB Direct (allowing for RDMA based network transfer, which does bypasses the TCP stack and moving data from memory to memory which gives low overhead, low latency connections.
Now both these features allow us to leverage better NIC utilization, but is aimed for a traditional configuration where storage is still a seperate resource from computing. My guess is that when we are going to deploy a Storage Spaces Direct cluster on Windows Server 2016 in a HCI deployment these features will be disabled.
So how does Nutanix work with SMB 3 ?
First of, important to understand the underlaying structure of the Nutanix OS. First of all local storage in the Nutanix nodes from a cluster are added to a unified pool of storage which are part of the Nutanix distributed filesystem. On top of this we create containers which have their settings like compression, dedup and replication factor which defines the amount of copies of data within a container. The reason for these copies are for fault-tolerance in case of a node failure or disk failure. So in essence you can think about this is a DAG (Database availability Groups) but for virtual machines.
So for SMB we can have shares which are represented as containers which again are created on top of a Nutanix cluster. Which are then presented to the Hyper-V hosts for VM placement.
Also important to remember that even thou we have a distributed file system across different nodes, the data is always run locally for a node (reason for this is so that the network does not becoming a point of congestion) Nutanix has a special role called the Curator (Which runs on the CVM)which is responsible for moving the hot data as local to the VM as possible. So if we for instance do a migration from host 1 to host 2, the CVM on host 1 might still contain the VM data and then reads and writes will from host 2 to CVM on host 1 the CVM will start to cache the data locally.
Now since this architecture leverages data locallity there is no need for feature like SMB Direct and SMB multichannel so therefore these features are not required in a Nutanix deployment for Hyper-V, however is does support SMB transparent failover which allows for continuously available file shares.
Now I haven’t started to explain yet how this architecture handles I/O yet, this is where the magic happens. Stay tuned.