Difference between revisions of "Setup an Interactive Queue"

From Define Wiki
Jump to navigation Jump to search
(Created page with "This example covers adding IB as a Boolean resource to LSF: Type Boolean: Either a nodes has it (1) or doesnt (0) ===== Add the Resource to LSF ===== * In file /etc/cfm/te...")
 
 
(One intermediate revision by the same user not shown)
Line 1: Line 1:
This example covers adding IB as a Boolean resource to LSF:
+
===== Setup the queue =====
 +
* lsb.queues file
 +
* once updated, run addhost -u to sync
 +
* bqueues to check the available queues
 +
<syntaxhighlight>
 +
# Example conf, administrators/priority/hosts will differ from cluster to cluster
 +
Begin Queue
 +
QUEUE_NAME=interactive
 +
DESCRIPTION=default for interactive jobs
 +
ADMINISTRATORS=user2 userGroupA
 +
PRIORITY=80
 +
INTERACTIVE=ONLY
 +
NEW_JOB_SCHED_DELAY=0
 +
HOSTS=hostGroupB+5 hostGroupA+2 others
 +
End Queue
 +
</syntaxhighlight>
  
Type Boolean: Either a nodes has it (1) or doesnt (0)
+
===== Submit an interactive job =====
 +
<syntaxhighlight>
 +
bsub -Ip -XF bash -l
 +
</syntaxhighlight>
  
 
+
===== X-forwarding not working =====
===== Add the Resource to LSF =====
+
* Ensure xorg-x11-xauth and xterm are installed
* In file /etc/cfm/templates/lsf/default.lsf.shared, add to following line to the '''RESOURCE''' stanza.
+
<syntaxhighlight>
<pre>
+
# use ngedit to select the following pacakges
Begin Resource
+
- xorg-x11-xauth
RESOURCENAME  TYPE    INTERVAL INCREASING  DESCRIPTION        # Keywords
+
- xterm
...
+
</syntaxhighlight>
ib            Boolean    ()    ()          (Infiniband Availability)
 
End Resource
 
</pre>
 
 
 
===== Assign Resource to Hosts =====
 
* In file /etc/cfm/templates/lsf/default.lsf.cluster, add the following after the '''XXX_lsfmc_XXX''' tag. NOTE: Nodes with the IB tag have to be listed individually.
 
<pre>
 
Begin  Host
 
HOSTNAME  model    type        server r1m  mem  swp  RESOURCES    #Keywords
 
...
 
XXX_lsfmc_XXX  !  !  1  3.5  ()  ()  (mg)
 
cn109  !      !      1      3.5    ()      ()      (openmpi mpich2 mvapich ib)
 
cn110  !      !      1      3.5    ()      ()      (openmpi mpich2 mvapich ib)
 
cn111  !      !      1      3.5    ()      ()      (openmpi mpich2 mvapich ib)
 
cn112  !      !      1      3.5    ()      ()      (openmpi mpich2 mvapich ib)
 
...
 
End Host
 
</pre>
 
 
 
===== Restart LSF =====
 
<pre>
 
addhost -u
 
</pre>
 
 
 
===== Submit a job requesting the resource =====
 
<pre>
 
#BSUB -R 'select[ib]'
 
 
 
Sample IMB Script
 
#BSUB -n 48
 
#BSUB -R 'select[ib]'
 
#BSUB -R 'span[ptile=12]'
 
#BSUB -R 'cu[maxcus=1]'
 
#BSUB -o IB.%J.out
 
#BSUB -e IB.%J.err
 
#BSUB -J v_IB
 
#BSUB -L /bin/bash
 
 
 
. /etc/profile
 
module load PMPI/modulefile
 
 
 
mpirun -prot -np $LSB_DJOB_NUMPROC /home/viglen/scratch/IMB_3.0/bin/imb-pmpi
 
</pre>
 

Latest revision as of 10:04, 13 May 2013

Setup the queue
  • lsb.queues file
  • once updated, run addhost -u to sync
  • bqueues to check the available queues
# Example conf, administrators/priority/hosts will differ from cluster to cluster
Begin Queue
QUEUE_NAME=interactive
DESCRIPTION=default for interactive jobs
ADMINISTRATORS=user2 userGroupA
PRIORITY=80
INTERACTIVE=ONLY
NEW_JOB_SCHED_DELAY=0
HOSTS=hostGroupB+5 hostGroupA+2 others
End Queue
Submit an interactive job
bsub -Ip -XF bash -l
X-forwarding not working
  • Ensure xorg-x11-xauth and xterm are installed
# use ngedit to select the following pacakges 
- xorg-x11-xauth
- xterm