Difference between revisions of "CloudX: Mellanox CloudX Installation"

From Define Wiki
Jump to navigation Jump to search
Line 49: Line 49:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
−
=== Setup Control Node with KVM ==
+
=== Setup Control Node with KVM ===
 
* Verify Host Supports KVM  
 
* Verify Host Supports KVM  
 
<syntaxhighlight>
 
<syntaxhighlight>
Line 98: Line 98:
 
BRIDGE=br0
 
BRIDGE=br0
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
* Reboot the node once complete
 +
<syntaxhighlight>
 +
reboot
 +
</syntaxhighlight>
 +
 +
=== Configure the CloudX VM ===
 +
* Bring up the KVM manager
 +
<syntaxhighlight>
 +
# X11 fwding required
 +
virt-manager
 +
</syntaxhighlight>
 +
 +
* Steps in the manager:
 +
# Step 1 of 4
 +
# Create a new VM
 +
# Select 'Import existing disk image'
 +
# Click 'Forward'
 +
# Step 2 of 4
 +
# Select the qcow2 image
 +
# OS Type: Linux
 +
# Version: Redhat Enterprise Linux 6
 +
# Select 'Forward'
 +
# Step 3 of 4
 +
# RAM: 1024MB
 +
# CPUs: 1
 +
# Select 'Forward'
 +
# Step 4 of 4
 +
# Select the advanced options
 +
# Host device should be br0
 +
# Virt Type: KVM
 +
# Arch: x86_64
 +
# Finish
 +
 +
* Shut down the VM when it starts, we need to edit the disk format
 +
* Select the 'i' or Information tab, Select the disk option
 +
* Make sure the '''Storage Format''' is <tt>qcow2</tt>
 +
* Make sure the '''Disk Bus''' is <tt>IDE</tt>
 +
* Power on the VM and let it boot
 +
* Check the VM settings (IP DHCP etc, go through VNC)
 +
<syntaxhighlight>
 +
# Note; Keys were not working correctly through virt-manager, had to use vncviewer instead
 +
yum install tsclient
 +
</syntaxhighlight>
 +
 +
You can revert back to ssh once you have the IP address. The default username and password is '''root''' and '''password'''

Revision as of 12:49, 30 April 2014

Download the CloudX Image

# Standard
wget http://support.mellanox.com/ftp/versions/current/Solutions/cloudX/1.0.0.8/ONE_CLICK_CLOUDX_1.0.0.8-31032014-2146.qcow2
# Continue WHEN things go wrong
wget -c http://support.mellanox.com/ftp/versions/current/Solutions/cloudX/1.0.0.8/ONE_CLICK_CLOUDX_1.0.0.8-31032014-2146.qcow2

Base System Setup

  • CentOS 6.4 Base System
  • MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64.tgz package copied across

Package Setup

  # Install the additional packages
  yum install -y tcl gcc-gfortran.x86_64 tk 
  tar zxvf MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64.tgz 
  cd MLNX_OFED_LINUX-2.2-0.0.2_20140306_1723-rhel6.4-x86_64
  ./mlnxofedinstall --force --all

Setup Grub

  • Setup GRUB to boot with SR-IOV support, add intel_iommu=on to the kernel args
title CentOS (2.6.32-358.el6.x86_64)
        root (hd0,0)
        kernel /vmlinuz-2.6.32-358.el6.x86_64 ro root=/dev/mapper/vg_blade3-lv_root \
                  rd_NO_LUKS rd_LVM_LV=vg_blade3/lv_root LANG=en_US.UTF-8 \ 
                  rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=vg_blade3/lv_swap \ 
                  KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet intel_iommu=on
        initrd /initramfs-2.6.32-358.el6.x86_64.img
        • NOTE SETUP THE BIOS - come back once confirmed settings

Verify PCI Speeds

  • Setup PCI Utils, make sure we are running at 8GT/s
  • ConnectX2 Sample Output - This is only GEN2 output
[root@blade1 ~]# lspci -d 15b3: -vv | grep LnkSta
		LnkSta:	Speed 5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-

Setup Control Node with KVM

  • Verify Host Supports KVM
egrep '(vmx|svm)' --color=always /proc/cpuinfo
# Should have a vmx flag on each core
  • Install KVM
yum install xauth
yum groupinstall  Virtualisation 'Virtualization Client' 'Virtualization Platform' 'Virtualization Tools'
modprobe kvm kvm-intel 
/etc/init.d/libvirtd start 
chkconfig libvirtd on
  • Verify the virbr0 interface is setup and ready
[root@blade2 ~]# ifconfig virbr0 
virbr0    Link encap:Ethernet  HWaddr 52:54:00:1D:04:7A  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
  • Setup a network bridge on the host with DHCP
# create the following file
[root@blade2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-br0
DEVICE=br0 
TYPE=Bridge 
BOOTPROTO=dhcp 
ONBOOT=yes 
DELAY=0
  • Edit the eth0 configuration to add a bridge: BRIDGE=br0
[root@blade2 ~]# cat  /etc/sysconfig/network-scripts/ifcfg-eth0 
DEVICE=eth0
ONBOOT=yes
HWADDR=00:25:90:C4:E9:8A
TYPE=Ethernet
BOOTPROTO=dhcp
BRIDGE=br0
  • Reboot the node once complete
reboot

Configure the CloudX VM

  • Bring up the KVM manager
# X11 fwding required
virt-manager
  • Steps in the manager:
  1. Step 1 of 4
  2. Create a new VM
  3. Select 'Import existing disk image'
  4. Click 'Forward'
  5. Step 2 of 4
  6. Select the qcow2 image
  7. OS Type: Linux
  8. Version: Redhat Enterprise Linux 6
  9. Select 'Forward'
  10. Step 3 of 4
  11. RAM: 1024MB
  12. CPUs: 1
  13. Select 'Forward'
  14. Step 4 of 4
  15. Select the advanced options
  16. Host device should be br0
  17. Virt Type: KVM
  18. Arch: x86_64
  19. Finish
  • Shut down the VM when it starts, we need to edit the disk format
  • Select the 'i' or Information tab, Select the disk option
  • Make sure the Storage Format is qcow2
  • Make sure the Disk Bus is IDE
  • Power on the VM and let it boot
  • Check the VM settings (IP DHCP etc, go through VNC)
# Note; Keys were not working correctly through virt-manager, had to use vncviewer instead
yum install tsclient

You can revert back to ssh once you have the IP address. The default username and password is root and password

Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=CloudX:_Mellanox_CloudX_Installation&oldid=4377"