Difference between revisions of "VScaler: CIX Network Node Preparation"
Jump to navigation
Jump to search
| Line 36: | Line 36: | ||
* Install '''ibutils''' and '''infiniband-diags''' | * Install '''ibutils''' and '''infiniband-diags''' | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| − | [root@ | + | [root@network1 ~]# yum -y install ibutils infiniband-diags |
</syntaxhighlight> | </syntaxhighlight> | ||
* Stop and disable '''<code>NetworkManager</code>''' | * Stop and disable '''<code>NetworkManager</code>''' | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| − | [root@ | + | [root@network1 ~]# systemctl stop NetworkManager |
| − | [root@ | + | [root@network1 ~]# systemctl disable NetworkManager |
</syntaxhighlight> | </syntaxhighlight> | ||
* Bring up interface '''ib0''' | * Bring up interface '''ib0''' | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| − | [root@ | + | [root@network1 ~]# ifup ib0 |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
</syntaxhighlight> | </syntaxhighlight> | ||
* Check the IB link is up and showing 20Gb/s | * Check the IB link is up and showing 20Gb/s | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| − | [root@ | + | [root@network1 ~]# ibstat |
CA 'mthca0' | CA 'mthca0' | ||
CA type: MT25204 | CA type: MT25204 | ||
| Line 65: | Line 58: | ||
Firmware version: 1.2.0 | Firmware version: 1.2.0 | ||
Hardware version: a0 | Hardware version: a0 | ||
| − | Node GUID: | + | Node GUID: 0x0030487f35500000 |
| − | System image GUID: | + | System image GUID: 0x0030487f35500003 |
Port 1: | Port 1: | ||
| − | State: | + | State: Active |
Physical state: LinkUp | Physical state: LinkUp | ||
Rate: 20 | Rate: 20 | ||
| − | Base lid: | + | Base lid: 3 |
LMC: 0 | LMC: 0 | ||
| − | SM lid: | + | SM lid: 1 |
Capability mask: 0x02590a68 | Capability mask: 0x02590a68 | ||
| − | Port GUID: | + | Port GUID: 0x0030487f35500001 |
Link layer: InfiniBand | Link layer: InfiniBand | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Revision as of 11:27, 28 July 2016
Base OS Installation
- Base installation of CentOS 7.2 was performed from our PXE server (pxe.boston.co.uk)
- NB - This could have been done from the OHPC head node
- Nothing more than a vanilla installation
Set a hostname
[root@localhost ~]# hostnamectl set-hostname "network1.cix.vscaler.cloud" --static
[root@localhost ~]# hostname -f
network1.cix.vscaler.cloudConfigure the internal/management/provisioning network interface
- Create an
/etc/sysconfig/network-scripts/ifcfg-enp6s0f0file with the following contents
TYPE=Ethernet
BOOTPROTO=static
IPADDR=172.18.0.2
NETMASK=255.255.0.0
NAME=enp6s0f0
UUID=41d85fef-a915-4a89-a28a-a9cd4dac3de8
DEVICE=enp6s0f0
ONBOOT=yesConfigure the infiniband interface
- Create an
/etc/sysconfig/network-scripts/ifcfg-ib0file with the following contents
DEVICE=ib0
BOOTPROTO=static
IPADDR=172.20.0.2
NETMASK=255.255.0.0
ONBOOT=yes- Install ibutils and infiniband-diags
[root@network1 ~]# yum -y install ibutils infiniband-diags- Stop and disable
NetworkManager
[root@network1 ~]# systemctl stop NetworkManager
[root@network1 ~]# systemctl disable NetworkManager- Bring up interface ib0
[root@network1 ~]# ifup ib0- Check the IB link is up and showing 20Gb/s
[root@network1 ~]# ibstat
CA 'mthca0'
CA type: MT25204
Number of ports: 1
Firmware version: 1.2.0
Hardware version: a0
Node GUID: 0x0030487f35500000
System image GUID: 0x0030487f35500003
Port 1:
State: Active
Physical state: LinkUp
Rate: 20
Base lid: 3
LMC: 0
SM lid: 1
Capability mask: 0x02590a68
Port GUID: 0x0030487f35500001
Link layer: InfiniBandUpdate the /etc/hosts file
- It is necessary update the
/etc/hostsfile so that references to "controller" resolve to the local system
[root@head ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.18.0.1 controller head