Difference between revisions of "Pamcrash: Benchmark with Pamcrash"
Jump to navigation
Jump to search
| (5 intermediate revisions by 2 users not shown) | |||
| Line 16: | Line 16: | ||
# Copy the content of license file provides by ESI in 160708.flx | # Copy the content of license file provides by ESI in 160708.flx | ||
<nowiki> | <nowiki> | ||
| + | [user@blbdw1 flexlm]$ tar zxvf ../FLEXnet_v11.11.1_Linux-lsb3-x86_64.tar.gz | ||
| + | lmgrd | ||
| + | lmutil | ||
| + | pam_lmd | ||
../lmgrd -c 160708.flx > flexlm.log | ../lmgrd -c 160708.flx > flexlm.log | ||
| + | |||
| + | # you may need to install the following if you get an error about the ELF interpreter | ||
| + | yum install redhat-lsb -y | ||
</nowiki> | </nowiki> | ||
#.Check in flexlm.log if server is started and note if the port number | #.Check in flexlm.log if server is started and note if the port number | ||
| Line 22: | Line 29: | ||
== Set Licence server == | == Set Licence server == | ||
<nowiki> | <nowiki> | ||
| + | # for e.g. | ||
| + | # [user@blbdw1 flexlm]$ grep 'pam_lmd using' flexlm.log | ||
| + | # 22:07:03 (lmgrd) pam_lmd using TCP-port 56502 | ||
export PAMLIC=CRASHSAF | export PAMLIC=CRASHSAF | ||
| − | export PAM_LMD_LICENSE_FILE= | + | export PAM_LMD_LICENSE_FILE=56502@hostname |
| + | # (portnumber@hostname) | ||
</nowiki> | </nowiki> | ||
| Line 30: | Line 41: | ||
* As of Feb 2016, Platform MPI (pmpi) does not yet support psm2. To run pmpi on an OPA system we need to use psm2-compat. Furthermore, when PSM is selected, pmpi checks that the system has a True Scale HCA. So, we have to modify lspci to report a True Scale HCA. | * As of Feb 2016, Platform MPI (pmpi) does not yet support psm2. To run pmpi on an OPA system we need to use psm2-compat. Furthermore, when PSM is selected, pmpi checks that the system has a True Scale HCA. So, we have to modify lspci to report a True Scale HCA. | ||
<nowiki> | <nowiki> | ||
| + | # apply this to all nodes in the cluster | ||
mv /usr/sbin/lspci /usr/sbin/lspci_orig | mv /usr/sbin/lspci /usr/sbin/lspci_orig | ||
cat > /usr/sbin/lspci | cat > /usr/sbin/lspci | ||
| Line 49: | Line 61: | ||
</nowiki> | </nowiki> | ||
| − | + | '''NOTE:'''-mpiext is used to pass the option to platform mpi. | |
* Replace test.pc with appropriate file from VW | * Replace test.pc with appropriate file from VW | ||
Latest revision as of 23:25, 29 July 2016
Running Pamcrash with platform mpi
- Pamcrash comes precomplied with platform mpi , Though other mpi could be used as well
[root@head ~]# mkdir pamcrash [root@head ~]# cd pamcrash [root@head pamcrash]# tar -xvf VPSolution-2015.01_Linux-em64t.tar.bz2 [root@head pamcrash]# source setpamhome.bash /root/pamcrash/
- pamworld wrapper will located in /root/pamcrash/vpsolver/2015.01/
To run license server
- Copy the binaries lmgrd, pam_lmd and lmuti from ESI ftp folder.
- Copy the content of license file provides by ESI in 160708.flx
[user@blbdw1 flexlm]$ tar zxvf ../FLEXnet_v11.11.1_Linux-lsb3-x86_64.tar.gz lmgrd lmutil pam_lmd ../lmgrd -c 160708.flx > flexlm.log # you may need to install the following if you get an error about the ELF interpreter yum install redhat-lsb -y
- .Check in flexlm.log if server is started and note if the port number
Set Licence server
# for e.g. # [user@blbdw1 flexlm]$ grep 'pam_lmd using' flexlm.log # 22:07:03 (lmgrd) pam_lmd using TCP-port 56502 export PAMLIC=CRASHSAF export PAM_LMD_LICENSE_FILE=56502@hostname # (portnumber@hostname)
Running PAMCRASH over OPA with platform mpi
- As of Feb 2016, Platform MPI (pmpi) does not yet support psm2. To run pmpi on an OPA system we need to use psm2-compat. Furthermore, when PSM is selected, pmpi checks that the system has a True Scale HCA. So, we have to modify lspci to report a True Scale HCA.
# apply this to all nodes in the cluster mv /usr/sbin/lspci /usr/sbin/lspci_orig cat > /usr/sbin/lspci #!/bin/sh /usr/sbin/lspci_orig $@ echo "81:00.0 InfiniBand: QLogic Corp. IBA7322 QDR InfiniBand HCA (rev 02)" ^D pmpi.conf should contain: MPI_ICLIB_PSM__PSM_MAIN = /usr/lib64/psm2-compat/libpsm_infinipath.so.1 MPI_ICMOD_PSM__PSM_MAIN = "hfi1" MPI_ICLIB_PSM__PSM_PATH = /usr/lib64/psm2-compat/libpsm_infinipath.so.1 MPI_ICMOD_PSM__PSM_PATH = "hfi1" /root/pamcrash/vpsolver/2015.01/pamworld -np 24 -mpiext '-machinefile hosts -PSM' VW_UP05_adv2k_v12_noFPM_75.pc
NOTE:-mpiext is used to pass the option to platform mpi.
- Replace test.pc with appropriate file from VW
Running PAMCRASH with Infiniband
/root/pamcrash/vpsolver/2015.01/pamworld -np 24 -mpiext '-machinefile hosts -IBV' -arch em64t test.pc
NOTE: This uses ib verb for running. Can be force to run on -TCP as well with(IPoIB setup)