Pamcrash: Benchmark with Pamcrash

Running Pamcrash with platform mpi

• Pamcrash comes precomplied with platform mpi , Though other mpi could be used as well

[root@head ~]# mkdir pamcrash
[root@head ~]# cd pamcrash
[root@head pamcrash]# tar -xvf VPSolution-2015.01_Linux-em64t.tar.bz2
[root@head pamcrash]# source setpamhome.bash /root/pamcrash/  

• pamworld wrapper will located in /root/pamcrash/vpsolver/2015.01/

To run license server

1. Copy the binaries lmgrd, pam_lmd and lmuti from ESI ftp folder.
2. Copy the content of license file provides by ESI in 160708.flx

../lmgrd -c 160708.flx > flexlm.log

1. .Check in flexlm.log if server is started and note if the port number

Set Licence server

export PAMLIC=CRASHSAF
export PAM_LMD_LICENSE_FILE=34506@eno1 (portnumber@hostname)

Running PAMCRASH over OPA with platform mpi

• As of Feb 2016, Platform MPI (pmpi) does not yet support psm2. To run pmpi on an OPA system we need to use psm2-compat. Furthermore, when PSM is selected, pmpi checks that the system has a True Scale HCA. So, we have to modify lspci to report a True Scale HCA.

mv /usr/sbin/lspci /usr/sbin/lspci_orig
cat > /usr/sbin/lspci 
#!/bin/sh /usr/sbin/lspci_orig 
$@ echo "81:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)" 
^D 



pmpi.conf should contain:

MPI_ICLIB_PSM__PSM_MAIN = /usr/lib64/psm2-compat/libpsm_infinipath.so.1 
MPI_ICMOD_PSM__PSM_MAIN = "hfi1" 
MPI_ICLIB_PSM__PSM_PATH = /usr/lib64/psm2-compat/libpsm_infinipath.so.1
MPI_ICMOD_PSM__PSM_PATH = "hfi1" 


/root/pamcrash/vpsolver/2015.01/pamworld -np 24 -mpiext '-machinefile hosts -PSM' VW_UP05_adv2k_v12_noFPM_75.pc

'NOTE:'-mpiext is used to pass the option to platform mpi.

• Replace test.pc with appropriate file from VW

Running PAMCRASH with Infiniband

/root/pamcrash/vpsolver/2015.01/pamworld -np 24 -arch em64t test.pc