Windows2016

Windows 2016 server - Storage spaces Direct - Hyper converged install

• Install windows server 2016 Datacenter either with GUI or without. In this example i did with GUI but apparently you can use server core without issue. Not sure if you can use nano server. (Storage spaces direct is a Datacenter feature, this will not work or be licensed in anything less. For example Standard)

• Install hardware drivers and fully update OS. Make sure every node has the same update level otherwise cluster manager will complain. Plus network drivers will be required to get RDMA working (Storage spaces direct cannot function without RDMA)

• Tagged vlan will be required for RDMA. For my example i set a VLAN of 12 on the super micro switch on every port that its required and this is set to tagged. You can do this in the GUI or CLI of the swtich.

• Every node will need certain services. these can be installed using:

Install-WindowsFeature -Name File-Services
Install-WindowsFeature -Name Failover-Clustering -IncludeManagementTools
Install-WindowsFeature -Name Hyper-V -IncludeManagementTools -Restart

• Add all the nodes you wish to use for storage spaces direct to your active directory server. In my example i made a new 2016 AD server with a new forest

"win16.local" since i could never get adding to the cluster working on "bostonlabs.co.uk" This could be down to bostonlabs labs being 2008 R2 AD.

• Verify that the internal drives are online, by going to Server Manager > Tools > Computer

Management > Disk Management. If any are offline, select the drive, right-click it, and click Online. Alternatively, PowerShell can be used to bring all 14 drives in each host online with a single command.

• For the Mellanox NICs used in this solution, we need to enable Data Center Bridging (DCB),

which is required for RDMA. Then we create a policy to establish network Quality of Service (QoS) to ensure that the Software Defined Storage system has enough bandwidth to communicate between the nodes, ensuring resiliency and performance. We also need to disable regular Flow Control (Global Pause) on the Mellanox adapters, since Priority Flow Control (PFC) and Global Pause cannot operate together on the same interface.

1. Enable Data Center Bridging (required for RDMA)

Install-WindowsFeature -Name Data-Center-Bridging

1. Configure a QoS policy for SMB-Direct

New-NetQosPolicy "SMB" -NetDirectPortMatchCondition 445 -PriorityValue8021Action 3

1. Turn on Flow Control for SMB

Enable-NetQosFlowControl -Priority 3

1. Make sure flow control is off for other traffic

Disable-NetQosFlowControl -Priority 0,1,2,4,5,6,7

1. Apply a Quality of Service (QoS) policy to the target adapters

Enable-NetAdapterQos -Name "Mellanox 1","Mellanox 2"

1. Give SMB Direct a minimum bandwidth of 50%

New-NetQosTrafficClass "SMB" -Priority 3 -BandwidthPercentage 50 -Algorithm ETS

1. Disable Flow Control on physical adapters

Set-NetAdapterAdvancedProperty -Name "Mellanox 1" -RegistryKeyword "*FlowControl" -RegistryValue 0