Rocks: Adding an Infiniband Network on Rocks

Infiniband Script to add IPoIB to all nodes

Tested on Rocks 5.1

This assumes that OFED is correctly installed. The script below will add all the compute node's ib0 interface to the database and create /etc/sysconfig/network-scripts/ifcfg-ib0.

Edit the script below to reflect the number of nodes you are editing.

#!/bin/bash

rocks add network ipoib subnet=192.168.0.0 netmask=255.255.255.0;
echo "rocks add network ipoib subnet=192.168.0.0 netmask=255.255.255.0";

IP=100;
NODE=0;
NNODES=5

while [ $NODE -lt $NNODES ]; do  
        echo "Adding ib0 network to scarf-0-$NODE"
        rocks add host interface scarf-0-$NODE ib0 
        rocks set host interface ip scarf-0-$NODE iface=ib0 ip=192.168.0.$IP; 
        rocks set host interface subnet scarf-0-$NODE iface=ib0 subnet=ipoib; 
        rocks set host interface name scarf-0-$NODE iface=ib0 name=ib-0-$NODE;
    
        let IP++; 
        let NODE++; 
done;

echo "Sync config... ";
rocks sync config;

NODE=0;


while [ $NODE -lt $NNODES ]; do
        echo "Syncing network ib0 on scarf-0-$NODE...";
        rocks sync host network scarf-0-$NODE;

        let NODE++; 
done