Linux: Mellanox - Changing VPI from IB to Eth

• Tested on the blades (connectX2 with Centos 6.4)

Using the default Centos IB Support

  yum groupinstall 'Infiniband Support'
  yum install infiniband-diags opensm
  /etc/init.d/rdma restart

This will default to 2x IB ports.

• Check the PCI slots of the cards

[root@blade4 ~]# lspci | grep Mellanox
07:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

• Check the port 'personality' using the PCI slot from above

[root@blade4 ~]# cat /sys/bus/pci/devices/0000\:07\:00.0/mlx4_port1
ib
[root@blade4 ~]# cat /sys/bus/pci/devices/0000\:07\:00.0/mlx4_port2
ib

• Note: the OS provided mlx4_en module didnt support ethernet mode. Have to download updated modules: http://www.mellanox.com/page/software_overview_eth (Version at the time of writing this:http://www.mellanox.com/downloads/Drivers/mlnx-en-2.0-3.0.0.tgz

• Ensure the kernel-devel is present and use the ./install.sh provided with the mlnx_en package.

Using MLNX OFED

• Easy utility called connectx_port_config

[root@blade2 ~]# connectx_port_config 

ConnectX PCI devices :
|----------------------------|
| 1             0000:07:00.0 |
|----------------------------|

Before port change:
ib
eth

|----------------------------|
| Possible port modes:       |
| 1: Infiniband              |
| 2: Ethernet                |
| 3: AutoSense               |
|----------------------------|
Select mode for port 1 (1,2,3): 1
Select mode for port 2 (1,2,3): 2

After port change:
ib
eth