VScaler: CIX Controller Node Preparation

Base OS Installation

• Base installation of CentOS 7.2 was performed from our PXE server (pxe.boston.co.uk)
• Nothing more than a vanilla installation

Set a hostname

[root@localhost ~]# hostnamectl set-hostname "head.cix.vscaler.cloud" --static
[root@localhost ~]# hostname -f
head.cix.vscaler.cloud

Set up SSH keys

[root@head ~]# ssh-keygen -t dsa

Create sign on message (MOTD message)

• To create a MOTD message with dynamic information, create /etc/profile.d/motd.sh and populate with:

#!/bin/bash
echo -e "

#################################################################
#								#
#\e[32m             _____            __             ____  _____	\e[0m#
#\e[32m      _   __/ ___/_________ _/ /__  _____   / __ \/ ___/	\e[0m#
#\e[32m     | | / /\__ \/ ___/ __  / / _ \/ ___/  / / / /\__ \ 	\e[0m#
#\e[32m     | |/ /___/ / /__/ /_/ / /  __/ /     / /_/ /___/ / 	\e[0m#
#\e[32m     |___//____/\___/\__,_/_/\___/_/      \____//____/  	\e[0m#
#                                                   		#
#                                    				#
#################################################################

    Welcome to: \e[1;97;41m`hostname`\e[0m
    This system is running: \e[33m`cat /etc/redhat-release`\e[0m
    Kernel is: \e[33m`uname -r`\e[0m

    You are logged in as: \e[1;97;44m`whoami`\e[0m

#################################################################
"

Configure the infiniband interface

• Create an /etc/sysconfig/network-scripts/ifcfg-ib0 file with the following contents

DEVICE=ib0
BOOTPROTO=static
IPADDR=172.20.0.1
NETMASK=255.255.0.0
ONBOOT=yes

• Install ibutils and infiniband-diags

[root@head ~]# yum -y install ibutils infiniband-diags

• Stop and disable NetworkManager

[root@head ~]# systemctl stop NetworkManager
[root@head ~]# systemctl disable NetworkManager

• Bring up interface ib0

[root@head ~]# ifup ib0

• Install, start and enable opensm

[root@head ~]# yum -y install opensm
[root@head ~]# systemctl start opensm
[root@head ~]# systemctl enable opensm

• Check the IB link is up and showing 20Gb/s

[root@head ~]# ibstat
CA 'mthca0'
	CA type: MT25204
	Number of ports: 1
	Firmware version: 1.2.0
	Hardware version: a0
	Node GUID: 0x0030487f354a0000
	System image GUID: 0x0030487f354a0003
	Port 1:
		State: Initializing
		Physical state: LinkUp
		Rate: 20
		Base lid: 0
		LMC: 0
		SM lid: 0
		Capability mask: 0x02590a68
		Port GUID: 0x0030487f354a0001
		Link layer: InfiniBand

Update the /etc/hosts file

• It is necessary update the /etc/hosts file so that references to "controller" resolve to the local system

[root@head ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

172.18.0.1  controller head