Pamcrash: Benchmark with Pamcrash
Running Pamcrash with platform mpi
- Pamcrash comes precomplied with platform mpi , Though other mpi could be used as well
[root@head ~]# mkdir pamcrash [root@head ~]# cd pamcrash [root@head pamcrash]# tar -xvf VPSolution-2015.01_Linux-em64t.tar.bz2 [root@head pamcrash]# source setpamhome.bash /root/pamcrash/
- pamworld wrapper will located in /root/pamcrash/vpsolver/2015.01/
To run license server
- Copy the binaries lmgrd, pam_lmd and lmuti from ESI ftp folder.
- Copy the content of license file provides by ESI in 160708.flx
[user@blbdw1 flexlm]$ tar zxvf ../FLEXnet_v11.11.1_Linux-lsb3-x86_64.tar.gz lmgrd lmutil pam_lmd ../lmgrd -c 160708.flx > flexlm.log # you may need to install the following if you get an error about the ELF interpreter yum install redhat-lsb -y
- .Check in flexlm.log if server is started and note if the port number
Set Licence server
export PAMLIC=CRASHSAF export PAM_LMD_LICENSE_FILE=34506@eno1 (portnumber@hostname)
Running PAMCRASH over OPA with platform mpi
- As of Feb 2016, Platform MPI (pmpi) does not yet support psm2. To run pmpi on an OPA system we need to use psm2-compat. Furthermore, when PSM is selected, pmpi checks that the system has a True Scale HCA. So, we have to modify lspci to report a True Scale HCA.
mv /usr/sbin/lspci /usr/sbin/lspci_orig cat > /usr/sbin/lspci #!/bin/sh /usr/sbin/lspci_orig $@ echo "81:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)" ^D pmpi.conf should contain: MPI_ICLIB_PSM__PSM_MAIN = /usr/lib64/psm2-compat/libpsm_infinipath.so.1 MPI_ICMOD_PSM__PSM_MAIN = "hfi1" MPI_ICLIB_PSM__PSM_PATH = /usr/lib64/psm2-compat/libpsm_infinipath.so.1 MPI_ICMOD_PSM__PSM_PATH = "hfi1" /root/pamcrash/vpsolver/2015.01/pamworld -np 24 -mpiext '-machinefile hosts -PSM' VW_UP05_adv2k_v12_noFPM_75.pc
NOTE:-mpiext is used to pass the option to platform mpi.
- Replace test.pc with appropriate file from VW
Running PAMCRASH with Infiniband
/root/pamcrash/vpsolver/2015.01/pamworld -np 24 -mpiext '-machinefile hosts -IBV' -arch em64t test.pc
NOTE: This uses ib verb for running. Can be force to run on -TCP as well with(IPoIB setup)