View source for Install CUDA 4.x on PCM 2.0.1/2.1

==== Install CUDA 4.x on PCM 2.0.1/2.1 ====

'''Note:'''
<syntaxhighlight>
There are no CUDA 4.x kit for PCM 2.0.1/2.1. CUDA kit 4.x only available on HPC 3.0 onwards.

Therefore, CUDA 4.x will need to be install manually and merge Platform CUDA LSF kits config
into the existing config on PCM 2.0.1/2.1 
</syntaxhighlight>

* '''GPU''' node: Install cuda 4.0 devdirver 
<syntaxhighlight>
/shared/viglen/4.0/devdriver_4.0_linux_64_270.41.19.run --accept-license --no-questions --silent > /tmp/inst-cuda-driver.log 2>&1
</syntaxhighlight>

* '''GPU''' node: Install "Viglen build" cuda 4.x rpm (see how to create rpm below)
<syntaxhighlight>
rpm -iv --force --nodeps /shared/viglen/4.0/cuda-4.0-1.x86_64.rpm > /tmp/inst-cuda40-rpm.log 2>&1
</syntaxhighlight>

* '''GPU''' node: Untar 'lsf7Update6_linux2.6-glibc2.3-x86_64-157853.tar' to /opt/lsf/7.0
<syntaxhighlight>
cd /opt/lsf/7.0
tar xvf /shared/viglen/lsf7Update6_linux2.6-glibc2.3-x86_64-157853.tar > /tmp/inst-lsf7Update6.log 2>&1
</syntaxhighlight>

* '''Headnode''' node: extract nvjob, elim.nvidia and related files from platform-lsf-gpu-1.0.2.rpm 
<syntaxhighlight>
'Note': platform-lsf-gpu-1.0.1.rpm:- nvjob + elim.nvidia from this rpm will cause core dump.

- rpm2cpio platform-lsf-gpu-1.0.2.rpm > platform-lsf-gpu-1.0.2.cpio
- cd /tmp
- cpio -idmuv -I platform-lsf-gpu-1.0.2.cpio
- cp -r /tmp/opt/lsf_gpu/bin /shared/viglen/lsf-gpu
- cp -r /tmp/opt/lsf_gpu/etc /shared/viglen/lsf-gpu
</syntaxhighlight>

* '''GPU''' node: Replace the existing nvjob+elim.nvidia and its related files
<syntaxhighlight>
mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/nvjob /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin/nvjob.ORIG
cp /shared/viglen/lsf-gpu/bin/nvjob /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/bin


mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/elim.nvidia /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/elim.nvidia.ORIG
cp /shared/viglen/lsf-gpu/etc/elim.nvidia /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc


mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl2_p /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl2_p.ORIG
cp /shared/viglen/lsf-gpu/etc/esub.gpuexcl2_p /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc


mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl2_t /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl2_t.ORIG
cp /shared/viglen/lsf-gpu/etc/esub.gpuexcl2_t /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc


mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl_p /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl_p.ORIG
cp /shared/viglen/lsf-gpu/etc/esub.gpuexcl_p /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc


mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl_t /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpuexcl_t.ORIG
cp /shared/viglen/lsf-gpu/etc/esub.gpuexcl_t /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc


mv /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpushared /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc/esub.gpushared.ORIG
cp /shared/viglen/lsf-gpu/etc/esub.gpushared /opt/lsf/7.0/linux2.6-glibc2.3-x86_64/etc
</syntaxhighlight>

* '''Headnode:''' Merge lsf.cluster, lsf.shared and lsf.applications from 'platform-lsf-gpu-1.0-2.x86_64.rpm' into the existing config
<syntaxhighlight>
d=`date +%Y%m%d%H%M`
cp -p /etc/cfm/templates/lsf/default.lsf.cluster /etc/cfm/templates/lsf/default.lsf.cluster.${d}
cat lsf.cluster >> /etc/cfm/templates/default.lsf.cluster

cp -p /etc/cfm/templates/lsf/default.lsf.shared /etc/cfm/templates/lsf/default.lsf.shared.${d}
cat lsf.shared >> /etc/cfm/templates/default.lsf.shared

cp -p /opt/lsf/conf/lsbatch/headnode_cluster1/configdir/lsb.applications /opt/lsf/conf/lsbatch/headnode_cluster1/configdir/lsb.applications.${s}
cat lsb.applications >> /opt/lsf/conf/lsbatch/headnode_cluster1/configdir/lsb.applications

</syntaxhighlight>

* '''Headnode''': Add new CUDA Path to /etc/cfm/<gpu_ng>/etc/profile.d/cuda.sh (copied from the gpu node itself)
<syntaxhighlight>
export PATH=/usr/bin:/usr/sbin:/sbin:.:/pcc/cnbuild/build/bldcluster/7.0/linux2.6-glibc2.3-x86_64/etc:\
/pcc/cnbuild/build/bldcluster/7.0/linux2.6-glibc2.3-x86_64/bin:\
/home/cnbuild/bin:/pcc/saqa/build/scripts:/pcc/saqa/build/scripts/utils:\
/pcc/lsfqa-trusted/3rdparty/ant/apache-ant-1.6.0/bin:\
/pcc/lsfqa-trusted/3rdparty/jdk/1.5.0_08/linux-x86/bin:/usr/bin:/usr/local/bin:\
/usr/lib64/qt-3.3/bin:/usr/kerberos/bin:/usr/bin:/bin:/usr/local/bin:.:\
/pcc/lsfqa-trusted/symphony_ext/ant/1.5.1/bin:/pcc/saqa/build/scripts:\
/pcc/saqa/build/scripts/utils:/local/jdk1.4/bin:/usr/local/oracle/8.0.4/bin:\
/usr/bin:/usr/local/bin:/local/share/bin:/opt/local/bin:/bin:/sbin:/usr/sbin:\
/usr/ccs/bin:/etc:/usr/etc:/pcc/qa/ddts/bin:/opt/SUNWhpc/bin:/opt/SUNWspro/bin:\
/usr/openwin/bin:/opt/gnu/bin:/opt/ansic/bin:/opt/langtools/bin:/usr/softbench/bin:\
/usr/bin/X11:/usr/bsd:/local/bin/X11:/usr/hosts:/usr/vue/bin:/usr/ucb:/usr/sfw/bin:/sbin:\
/opt/cuda/4.0/toolkit/bin:/opt/cuda/4.0/sdk/C/bin/linux/release
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cuda/4.0/toolkit/lib64:/opt/cuda/4.0/toolkit/lib

</syntaxhighlight>

**Create CUDA 4.x rpm
<syntaxhighlight>
Note to come ...
</syntaxhighlight>