Difference between revisions of "Bright:infiniband"
| Line 6: | Line 6: | ||
== Installing OFED software == | == Installing OFED software == | ||
| + | |||
| + | A new version of OFED can be installed across the entire cluster, or to specific software images. | ||
| + | |||
| + | |||
| + | == Subnet Managers == | ||
| + | |||
| + | Every IB subnet requires an instance of the subnet manager to be running. The subnet manager manages the routing, addressing and initialization of the IB fabric. In some cases teh subnet manager may be running on the switch. However on older switches it will need to be started one of the systems on the subnet. On a bright cluster managing where the subnet manager is running should be done through cmdaemon. | ||
| + | |||
| + | There are two ways to do this: | ||
| + | |||
| + | <syntaxhighlight> | ||
| + | cmsh -c "device roles <node>; assign subnetnmanager; commit" | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | <syntaxhighlight> | ||
| + | [root@bright61 ~]# cmsh | ||
| + | [bright61]% device services node001 | ||
| + | [bright61->device[node001]->services]% add opensm | ||
| + | [bright61->device[node001]->services*[opensm*]]% set autostart yes | ||
| + | [bright61->device[node001]->services*[opensm*]]% set monitored yes | ||
| + | [bright61->device[node001]->services*[opensm*]]% commit | ||
| + | [bright61->device[node001]->services[opensm]]% | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | Through the GUI this can be achieved by assinging the "Subnet Manager Role" to the node. | ||
Latest revision as of 14:30, 10 February 2014
Standard OFED Software included with Bight
On a standard Bright install the OFED packages provided as part of the Linux distro are used. The required packages are installed by default on the head node and all software images.
The standard OFED packages can be replaced by vendor specific version either through the Bright Repos or a custom install.
Installing OFED software
A new version of OFED can be installed across the entire cluster, or to specific software images.
Subnet Managers
Every IB subnet requires an instance of the subnet manager to be running. The subnet manager manages the routing, addressing and initialization of the IB fabric. In some cases teh subnet manager may be running on the switch. However on older switches it will need to be started one of the systems on the subnet. On a bright cluster managing where the subnet manager is running should be done through cmdaemon.
There are two ways to do this:
cmsh -c "device roles <node>; assign subnetnmanager; commit"[root@bright61 ~]# cmsh
[bright61]% device services node001
[bright61->device[node001]->services]% add opensm
[bright61->device[node001]->services*[opensm*]]% set autostart yes
[bright61->device[node001]->services*[opensm*]]% set monitored yes
[bright61->device[node001]->services*[opensm*]]% commit
[bright61->device[node001]->services[opensm]]%Through the GUI this can be achieved by assinging the "Subnet Manager Role" to the node.