OpenHPC: Setup Intel Cluster Checker in OpenHPC

• Packages used to be installable from the repo but Intel removed it - gah! Instructions below are for manual setup
• Updated License Key: CMD9-H9684BWD (May 18)

Download/Install latest clck

• Check the intel registration centre for the latest version: https://registrationcenter.intel.com/

 wget http://registrationcenter-download.intel.com/akdlm/irc_nas/tec/9701/l_clck_p_2017.0.014.tgz 
 cd l_clck_p_2017.0.014
 ./install.sh
  
# step 1 serial number: enter CSKW-56T6M5ZD (valid to feb 2017)
# select components and installation params, default is fine. 
# Or install in /opt/ohpc/pub/intel as an NFS install - then ignore below

# lets add rpms for the compute image
 pwd
 /root/scratch/l_clck_p_2017.0.014
 rpm --root $CHROOT -ivh rpm/*rpm 

# hack - need to make the dirs similar on compute / head. Must be something dont by the install script rather than just the RPMs
 chroot $CHROOT
 cd /opt/intel/clck
 mv 2017.0.014/* 2017.0/
 rmdir 2017.0.014/
 ln -s 2017.0/ 2017.0.014

# open up ports so the collect works
firewall-cmd --permanent --zone=internal --add-port=49152/tcp

# and finally recreate the VNFS 
 wwvnfs -y --chroot $CHROOT

Run the cluster checker tool

• Setup the environment

# source the vars
 [boston@head1 clck_sanger]$ grep clck ~/.bashrc 
 source /opt/intel/clck_latest/bin/clckvars.sh 

# create the nodefile 
 for i in {01..20}; do echo node${i} >> nodefile ; done

• Run for the first time

clck-collect -a -f nodefile # this take a god damn eternity!
clck-analyze -f nodefile

Run quick perf test

# check memory bandwidth  (-I is include_only) 
clck -f ./nodefile -I stream -v 5

Using version 2.2

• This version provides us with the output we can use for accpetance test reporting

# URL may change, check registration centre: 
wget http://registrationcenter-download.intel.com/akdlm/irc_nas/tec/7754/l_clck_p_2.2.2.004.tgz
tar zxvf l_clck_p_2.2.2.004.tgz
cd l_clck_p_2.2.2.004
./install
# change the install dir to /opt/ohpc/pub/intel/clck/2.2.2.004

# now lets add the runtimes:
wget http://registrationcenter-download.intel.com/akdlm/irc_nas/tec/3506/intel_cluster_runtimes_3.6-1.tgz
yum install lsb libstdc++.i686
tar zxvf intel_cluster_runtimes_3.6-1.tgz
cd intel_cluster_runtimes_3.6-1
./install

# and again in the chroot
cp intel_cluster_runtimes_3.6-1.tgz /opt/ohpc/admin/images/centos7.2/root/scratch/
chroot $CHROOT
cd root/scratch 
yum install lsb libstdc++.i686
tar zxvf intel_cluster_runtimes_3.6-1.tgz
cd intel_cluster_runtimes_3.6-1
./install

• Then we need to setup the environment

mkdir /etc/intel
mkdir /opt/ohpc/admin/images/centos7.2/etc/intel
echo "CLUSTER_READY_VERSION=1.3.1" > /etc/intel/icr
echo "CLUSTER_READY_VERSION=1.3.1" > /opt/ohpc/admin/images/centos7.2/etc/intel/icr

• At this stage: reboot / reinstall nodes

Create a hosts file

You need to add a headnode to the hostsfile

[biosit@fotres bin]$ cat nodefile
fotres #type: head
node01
node02
node03
node04

Create a hosts file and configuration file

Add libraries location if you use different location

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/biosit/cluster_checker2.2/compiler/lib/intel64/

# From OpenHPC, put in the ~/.bashrc of a standard user
export LD_LIBRARY_PATH=/opt/intel/composer_xe_2013_sp1.0.080/compiler/lib/intel64/:$LD_LIBRARY_PATH
source /opt/intel/clck/2.2.2.004/bin/clckvars.sh
source /opt/intel/impi/4.1.1.036/bin64/mpivars.sh

# Copy the default configuration file: 
cp /opt/intel/clck/2.2.2.004/share/doc/intel-clck/defaults.xml ./cluster-conf.xml

Run Perf Checks for Version 2.2

# Network / IMB 
/opt/ohpc/pub/intel/clck/2.2.2.004/clck/2.2.2.004/bin/clck -c ./cluster-conf.xml --include_only imb_pingpong --verbose 5 --debug

# Linpack CPU Floating 
/opt/ohpc/pub/intel/clck/2.2.2.004/clck/2.2.2.004/bin/clck -c ./cluster-conf.xml --include_only dgemm --verbose 5 --debug

# Stream 
/opt/ohpc/pub/intel/clck/2.2.2.004/clck/2.2.2.004/bin/clck -c ./cluster-conf.xml --include_only stream --verbose 5 --debug

# Disk Bandwidth
/opt/ohpc/pub/intel/clck/2.2.2.004/clck/2.2.2.004/bin/clck -c ./cluster-conf.xml --include_only disk_bandwidth --verbose 5 --debug