Difference between revisions of "CloudX: Mellanox CloudX Installation"
Jump to navigation
Jump to search
| Line 16: | Line 16: | ||
* CentOS 6.4 Base System | * CentOS 6.4 Base System | ||
* MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64.tgz package copied across | * MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64.tgz package copied across | ||
| + | |||
| + | === Package Setup === | ||
<syntaxhighlight> | <syntaxhighlight> | ||
# Install the additional packages | # Install the additional packages | ||
| Line 24: | Line 26: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
| + | === Setup Grub === | ||
* Setup GRUB to boot with SR-IOV support, add <tt>intel_iommu=on</tt> to the <tt>kernel</tt> args | * Setup GRUB to boot with SR-IOV support, add <tt>intel_iommu=on</tt> to the <tt>kernel</tt> args | ||
<syntaxhighlight> | <syntaxhighlight> | ||
| Line 33: | Line 36: | ||
KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet intel_iommu=on | KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet intel_iommu=on | ||
initrd /initramfs-2.6.32-358.el6.x86_64.img | initrd /initramfs-2.6.32-358.el6.x86_64.img | ||
| + | </syntaxhighlight> | ||
| + | |||
| + | **** NOTE SETUP THE BIOS - come back once confirmed settings | ||
| + | === Verify PCI Speeds === | ||
| + | * Setup PCI Utils, make sure we are running at 8GT/s | ||
| + | |||
| + | * ConnectX2 Sample Output - '''This is only GEN2 output''' | ||
| + | <syntaxhighlight> | ||
| + | [root@blade1 ~]# lspci -d 15b3: -vv | grep LnkSta | ||
| + | LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- | ||
| + | LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1- | ||
</syntaxhighlight> | </syntaxhighlight> | ||
Revision as of 11:58, 30 April 2014
- Assumptions: Installed Centos 6.4 base system, nothing else
- Access the software at:
- http://support.mellanox.com/SupportWeb/solutions/cloudx
- Username: cloudx
- Password: CloudXme
Download the CloudX Image
# Standard
wget http://support.mellanox.com/ftp/versions/current/Solutions/cloudX/1.0.0.8/ONE_CLICK_CLOUDX_1.0.0.8-31032014-2146.qcow2
# Continue WHEN things go wrong
wget -c http://support.mellanox.com/ftp/versions/current/Solutions/cloudX/1.0.0.8/ONE_CLICK_CLOUDX_1.0.0.8-31032014-2146.qcow2Base System Setup
- CentOS 6.4 Base System
- MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64.tgz package copied across
Package Setup
# Install the additional packages
yum install -y tcl gcc-gfortran.x86_64 tk
tar zxvf MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64.tgz
cd MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64
./mlnxofedinstall --force --allSetup Grub
- Setup GRUB to boot with SR-IOV support, add intel_iommu=on to the kernel args
title CentOS (2.6.32-358.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-358.el6.x86_64 ro root=/dev/mapper/vg_blade3-lv_root \
rd_NO_LUKS rd_LVM_LV=vg_blade3/lv_root LANG=en_US.UTF-8 \
rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_blade3/lv_swap \
KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet intel_iommu=on
initrd /initramfs-2.6.32-358.el6.x86_64.img- NOTE SETUP THE BIOS - come back once confirmed settings
Verify PCI Speeds
- Setup PCI Utils, make sure we are running at 8GT/s
- ConnectX2 Sample Output - This is only GEN2 output
[root@blade1 ~]# lspci -d 15b3: -vv | grep LnkSta
LnkSta: Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-