Michael: Created page with "This example covers adding IB as a Boolean resource to LSF: Type Boolean: Either a nodes has it (1) or doesnt (0) ===== Add the Resource to LSF ===== * In file /etc/cfm/te..."

2012-12-06T16:09:44Z

Created page with "This example covers adding IB as a Boolean resource to LSF: Type Boolean: Either a nodes has it (1) or doesnt (0) ===== Add the Resource to LSF ===== * In file /etc/cfm/te..."

New page

This example covers adding IB as a Boolean resource to LSF:

Type Boolean: Either a nodes has it (1) or doesnt (0)

===== Add the Resource to LSF =====
* In file /etc/cfm/templates/lsf/default.lsf.shared, add to following line to the '''RESOURCE''' stanza.
<pre>
Begin Resource
RESOURCENAME TYPE INTERVAL INCREASING DESCRIPTION # Keywords
...
ib Boolean () () (Infiniband Availability)
End Resource
</pre>

===== Assign Resource to Hosts =====
* In file /etc/cfm/templates/lsf/default.lsf.cluster, add the following after the '''XXX_lsfmc_XXX''' tag. NOTE: Nodes with the IB tag have to be listed individually.
<pre>
Begin Host
HOSTNAME model type server r1m mem swp RESOURCES #Keywords
...
XXX_lsfmc_XXX ! ! 1 3.5 () () (mg)
cn109 ! ! 1 3.5 () () (openmpi mpich2 mvapich ib)
cn110 ! ! 1 3.5 () () (openmpi mpich2 mvapich ib)
cn111 ! ! 1 3.5 () () (openmpi mpich2 mvapich ib)
cn112 ! ! 1 3.5 () () (openmpi mpich2 mvapich ib)
...
End Host
</pre>

===== Restart LSF =====
<pre>
addhost -u
</pre>

===== Submit a job requesting the resource =====
<pre>
#BSUB -R 'select[ib]'

Sample IMB Script
#BSUB -n 48
#BSUB -R 'select[ib]'
#BSUB -R 'span[ptile=12]'
#BSUB -R 'cu[maxcus=1]'
#BSUB -o IB.%J.out
#BSUB -e IB.%J.err
#BSUB -J v_IB
#BSUB -L /bin/bash

. /etc/profile
module load PMPI/modulefile

mpirun -prot -np $LSB_DJOB_NUMPROC /home/viglen/scratch/IMB_3.0/bin/imb-pmpi
</pre>

Specifying Resources for Nodes - Revision history

Michael: Created page with "This example covers adding IB as a Boolean resource to LSF: Type Boolean: Either a nodes has it (1) or doesnt (0) ===== Add the Resource to LSF ===== * In file /etc/cfm/te..."