View source for CUDA: Installing CUDA and Building the SDK

===== Cuda 1.1 on Rocks 4.3 =====
 
installed the sdk/toolkit and driver

Problem linking with -lglut, do the following:

<pre>
ln -s /usr/lib/libglut.so.3 /usr/lib/libglut.so
ln -s /usr/lib64/libglut.so.3 /usr/lib64/libglut.so
</pre>


===== Adding CUDA roll to installed headnode =====

<pre>
mount -o loop cuda-4.3.0-x86_64.disk1.iso /mnt/cdrom
rocks add roll
rocks enable roll cuda
rocks list roll

# then on the node you want to install on
cd /home/install/rolls/cuda/4.3/x86_64/RedHat/RPMS/

# load the sdk and toolkit
rpm -i cuda-* \ 
       /home/install/rolls/os/4.3/x86_64/RedHat/RPMS/freeglut-2.2.0-14.x86_64.rpm \
       /home/install/rolls/os/4.3/x86_64/RedHat/RPMS/xorg-x11-Mesa-libGLU-6.8.2-1.EL.18.x86_64.rpm

# load the nvidia driver
rpm -ivh nvidia-driver*

# needs kernel source on the node
yum install kernel-devel
</pre>


===== Centos/Rocks 5 =====
If installing on a headnode with no nvidia gpu
<pre>
./NVIDIA-Linux-x86_64-180.22-pkg2.run --no-kernel-module --accept-license --no-questions --silent
</pre>

Install the SDK and Toolkit to a shared directory
<pre>
mkdir /opt/cuda/toolkit/2.1
./cudatoolkit_2.1_linux64_rhel5.2.run -- -prefix=/opt/cuda/toolkit/2.1/

mkdir /opt/cuda/sdk/2.1
./cuda-sdk-linux-2.10.1215.2015-3233425.run -- -prefix=/opt/cuda/sdk/2.1/ -cudaprefix=/opt/cuda/toolkit/2.1/
</pre>

Build the sdk (g++ missing plus a few others)
<pre>
export PATH=/share/apps/cuda/toolkit/2.3/bin/:$PATH
export LD_LIBRARY_PATH=/share/apps/cuda/toolkit/2.3/lib64:$LD_LIBRARY_PATH 
export INCLUDE=/share/apps/cuda/toolkit/2.3/include/:$INCLUDE
export CUDA_INSTALL_PATH=/share/apps/cuda/toolkit/2.3

[or edit the common/common.mk to point at the right CUDA install dir]

yum install libXi-devel
yum install libXmu-devel
yum install gcc kernel-devel gcc-c++ freeglut.i386 freeglut.x86_64 freeglut-devel glibc-devel

# Had to do this for cuda 2.1, dont think it's necessary in newer versions
ln -s /usr/lib64/libGLU.so.1 /usr/lib64/libGLU.so
</pre>


===== Centos 5.4 - CUDA 2.3 - PSC =====
<pre>
0) Make sure you have internet access as you need additional libs and packages.
1) Install Centos 5.4 with full development system (include gcc-c++), kernel-devel, kernel-header (or yum install gcc gcc-c++ kernel-devel, kernel-header)
2) yum install freeglut freeglut-devel 
3) yum install libICE-devel libSM-devel libXi-devel libXmu-devel libXmu libXt-devel
4) yum install mesa-LibGLU-devel mesa-libGLU xorg-x11-util-macros
5) mkdir -p /opt/cuda/2.3/toolkit /opt/cuda/2.3/sdk
6) Stop Xserver if running
7) ./NVIDIA-Linux-x86_64_190.53-pkg2.run
8) ./cudatoolkit_2.3_linux_64_rhel5.3.run -- -prefix=/opt/cuda/2.3/toolkit
9) export PATH=$PATH:/opt/cuda/2.3/toolkit/bin
   export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cuda/2.3/toolkit/lib64
   export INCLUDE=/opt/cuda/2.3/toolkit/include/:$INCLUDE
   export CUDA_INSTALL_PATH=/opt/cuda/2.3/toolkit
10)./cudasdk_2.3_linux.run -- -prefix=/opt/cuda/2.3/sdk -cudaprefix=/opt/cuda/2.3/toolkit
11) cd /opt/cuda/2.3/sdk/C
12) Build sdk sampe program:
    make
13) Check GPU device: /opt/cuda/2.3/sdk/C/bin/linux/release/deviceQuery --noprompt
</pre>

14) Change display setting (/etc/X11/xorg.conf)- May differ for different monitor
<pre>
Section "Monitor"
    HorizSync    31.0 - 65.0
    VertRefresh  50.0 - 60.0

Section "Screen"
   SubSection "Display"
     Modes "1280x1024" "1024x768" "800x600"

</pre>

===== OpenSUSE 11.1 - CUDA 2.3 -PSC =====

zypper install freeglut-devel

<pre>
./cudadriver_2.3_linux_64_190.18.run
OR
./cudadriver_2.3_linux_64_190.18.run --accept-license --no-questions --silent (not tested)
</pre>

Display setting (/etc/X11/xorg.conf)
<pre>
Section "Monitor"
    HorizSync    31.0 - 65.0
    VertRefresh  50.0 - 60.0

Section "Screen"
   SubSection "Display"
     Modes "1280x1024" "1024x768" "800x600"

</pre>

Install the SDK and Toolkit to a shared directory
<pre>
mkdir /opt/cuda/toolkit/2.3
./cudatoolkit_2.3_linux64_suse11.run -- -prefix=/opt/cuda/toolkit/2.1/

export PATH=$PATH:/opt/cuda/toolkit/2.3/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cuda/toolkit/2.3/lib64
export INCLUDE=/opt/cuda/toolkit/2.3/include/:$INCLUDE
export CUDA_INSTALL_PATH=/opt/cuda/toolkit/2.3

mkdir /opt/cuda/sdk/2.3
./cudasdk_2.3_linux.run -- -prefix=/opt/cuda/sdk/2.3/ -cudaprefix=/opt/cuda/toolkit/2.3/

</pre>

Build SDK 
<pre>
cd /opt/cuda/sdk/2.3/C
make
</pre>

===== Ubuntu 8.04.2 =====
<pre>
Note: 
CUDA 2.1 & ubuntu 8.04.2 have problem with top manubar nevigation.

To wrokaround the problem, Change (line 79/80) /usr/bin/compiz
From: INDIRECT="no"
To  : INDIRECT="yes"

Also, nvidia-xconfig from CUDA 2.1 does not generate xorg.conf correctly
whereas CUDA 2.0 does. Need to set the VertSync & HorSync manually.

1) Install Ubuntu 8.04.2 and make sure it can access the internet.
2) sudo apt-get install ssh
3) Put LiveCD to drive and 'sudo apt-cdrom add'
4) sudo apt-get update
5) sudo apt-get install build-essential
6) sudo apt-get install --resinstall mesa-common-dev
7) sudo apt-get libx11-dev freeglut3-dev freeglut3
8) sudo apt-get install libxi-dev
9) sudo apt-get install libxmu-dev
10) sudo apt-get install libc6-dev-i386
11) sudo apt-get remove nvidia-kernel-common
12) sudo ./NVIDIA-Linux-x86_64-180.22-pkg2.run
13) sudo ./cudatoolkit_2.1_linux64_ubuntu8.04.run
14) sudo ./cuda-sdk-linux-2.10.1215.2015-3233425.run 
15) cd /usr/local/NVIDIA_CUDA_SDK; sudo make
16) add 'PATH=$PATH:/usr/local/cuda/bin:/usr/local/NVIDIA_CUDA_SDK/bin/linux/release; export PATH' 
    to /etc/skel/.bashrc or $HOME/.bashrc
17) add 'LD_LIBRARY_PATH=/usr/local/cuda/lib; export LD_LIBRARY_PATH' 
    to /etc/skel/.bashrc or $HOME/.bashrc
18) add /etc/X11/xorg.conf:
    ==> Section Monitor"
         HorizSync 31.0 - 65.0
         VertRefresh 51.0 - 90.0
        EndSubSection

    ==> SubSection "Display"
         Depth 24
         Modes "1280x1024_60"
        EndSubSection
19) Checks CUDA /usr/local/NVIDIA_CUDA_SDK/bin/linux/release/deviceQuery
</pre>


===== Ubuntu 9.04 and CUDA 2.3 =====
<pre>
# ubuntu 9.04 and cuda 2.3
ctrl + alt + f1
login
sudo /etc/init.d/gdm stop
cd /media/disk (dir with the CUDA apps)

# install driver
sudo ./NVIDIA-Linux-x86_64-190.53-pkg2.run
Q. Run nvidia-xconfig utility (yes)

# install toolkit
./cudatoolkit_2.3_linux_64_ubuntu9.04.run
go with default [/usr/local/cuda]

# install sdk
./cudasdk_2.3_linux.run # no sudo!

# setup environment, edit ~/.bashrc
# append the following:
export PATH=$PATH:/usr/local/cuda/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64

# additional packages to build SDK
sudo apt-get install build-essential libglut3-dev libxmu-dev libxmu-headers libxi-dev

# build the sdk
cd ~/NVIDIA_GPU_Computing_SDK/C
make

At this stage, best off to reboot the system gdm wouldnt restart correctly

# check the sdk
login, open up an xterm [alt+f2, xterm]
cd NVIDIA_GPU_Computing_SDK/C/bin/linux/release
./deviceQuery [should show all the nvidia cards connected]
./nbody [make sure the graphical display is running ok]
</pre>

===== Ubuntu 9.10 and CUDA 2.3 =====
<pre>
# 9.10 has gcc 4.4 which does not work with CUDA 2.3
# If someone updates to 9.10 run the following commands:
sudo update-alternatives /usr/bin/gcc gcc /usr/bin/gcc-4.3 60
sudo update-alternatives /usr/bin/g++ g++ /usr/bin/g++-4.3 60
# install 4.3 if necessary
</pre>

===== Platform 3.0 / Centos 5.6 / Cuda 4.0 =====
<syntaxhighlight>
# Install all the required RPMs to build nvidia kernel and SDK 
yum install libXi-devel libXmu-devel gcc kernel-devel gcc-c++ freeglut.i386 freeglut.x86_64 freeglut-devel glibc-devel kernel-source

Resulted in:
(1/32): libXdmcp-devel-1.0.1-2.1.x86_64.rpm               | 7.5 kB     00:00
(2/32): xorg-x11-util-macros-1.0.2-4.fc6.x86_64.rpm       | 8.1 kB     00:00
(3/32): libSM-devel-1.0.1-3.1.x86_64.rpm                  | 9.4 kB     00:00
(4/32): libXau-devel-1.0.1-3.1.x86_64.rpm                 |  11 kB     00:00
(5/32): libICE-devel-1.0.1-2.1.x86_64.rpm                 |  13 kB     00:00
(6/32): libXxf86vm-1.0.1-3.1.i386.rpm                     |  14 kB     00:00
(7/32): libXxf86vm-1.0.1-3.1.x86_64.rpm                   |  14 kB     00:00
(8/32): libdrm-2.0.2-1.1.i386.rpm                         |  19 kB     00:00
(9/32): libdrm-2.0.2-1.1.x86_64.rpm                       |  19 kB     00:00
(10/32): libXmu-devel-1.0.2-5.i386.rpm                    |  21 kB     00:00
(11/32): libXmu-devel-1.0.2-5.x86_64.rpm                  |  21 kB     00:00
(12/32): libSM-1.0.1-3.1.i386.rpm                         |  27 kB     00:00
(13/32): libXi-devel-1.0.1-4.el5_4.x86_64.rpm             |  51 kB     00:00
(14/32): libXi-devel-1.0.1-4.el5_4.i386.rpm               |  51 kB     00:00
(15/32): libICE-1.0.1-2.1.i386.rpm                        |  54 kB     00:00
(16/32): libXmu-1.0.2-5.i386.rpm                          |  62 kB     00:00
(17/32): mesa-libGLU-devel-6.5.1-7.8.el5.x86_64.rpm       |  91 kB     00:00
(18/32): freeglut-devel-2.4.0-7.1.el5.i386.rpm            |  98 kB     00:00
(19/32): freeglut-devel-2.4.0-7.1.el5.x86_64.rpm          | 111 kB     00:00
(20/32): freeglut-2.4.0-7.1.el5.i386.rpm                  | 142 kB     00:00
(21/32): freeglut-2.4.0-7.1.el5.x86_64.rpm                | 150 kB     00:00
(22/32): libXt-1.0.2-3.2.el5.i386.rpm                     | 173 kB     00:00
(23/32): mesa-libGLU-6.5.1-7.8.el5.x86_64.rpm             | 224 kB     00:00
(24/32): mesa-libGLU-6.5.1-7.8.el5.i386.rpm               | 225 kB     00:00
(25/32): xorg-x11-proto-devel-7.1-13.el5.x86_64.rpm       | 247 kB     00:00
(26/32): libXt-devel-1.0.2-3.2.el5.x86_64.rpm             | 330 kB     00:00
(27/32): mesa-libGL-devel-6.5.1-7.8.el5.x86_64.rpm        | 461 kB     00:00
(28/32): libX11-devel-1.0.3-11.el5.x86_64.rpm             | 661 kB     00:00
(29/32): glibc-devel-2.5-58.i386.rpm                      | 2.0 MB     00:00
(30/32): kernel-devel-2.6.18-238.el5.x86_64.rpm           | 5.5 MB     00:00
(31/32): mesa-libGL-6.5.1-7.8.el5.x86_64.rpm              | 8.7 MB     00:00
(32/32): mesa-libGL-6.5.1-7.8.el5.i386.rpm                | 9.6 MB     00:00

# Note: Do checks the kernel, kernel-headers and kernel-devel are on the same version number
# after yum install kernel-headers and kernel-devel. If not, you will need to update the 
# "kernel" to the same version as kernel-devel and kernel-headers.

# Install the CUDA device driver (assuming node has GPU, otherwise include --no-kernel-module)
./devdriver_4.0_linux_64_270.41.19.run --accept-license --no-questions --silent

# Install the CUDA toolkit
mkdir -p /opt/cuda/4.0/toolkit
./cudatoolkit_4.0.17_linux_64_rhel5.5.run -- -prefix=/opt/cuda/4.0/toolkit
export PATH=$PATH:/opt/cuda/4.0/toolkit/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/cuda/4.0/toolkit/lib64:/opt/cuda/4.0/toolkit/lib
export CUDA_INSTALL_PATH=/opt/cuda/4.0/toolkit

# Install CUDA Tools
mkdir -p /opt/cuda/4.0/cudatools
./cudatools_4.0.17_linux_64.run -- -prefix=/opt/cuda/4.0/cudatools

# Install CUDA SDK
mkdir -p /opt/cuda/4.0/sdk
./gpucomputingsdk_4.0.17_linux.run -- -prefix=/opt/cuda/4.0/sdk/ -cudaprefix=/opt/cuda/4.0/toolkit/

# Build CUDA SDK
cd /opt/cuda/4.0/sdk
make
export PATH=$PATH:/opt/cuda/4.0/sdk/C/bin/linux/release/
</syntaxhighlight>

===== Installing from Kit with Platform 3.0.1/3.1 =====

'''Note:''' The kit does not appear to isntall the SKD toolkit automatically so be sure to add the following to your custom scripts

<syntaxhighlight>

# Dependencies needed if not already install before compiling GPU SDK (RHEL 6.2)
# in RHEL 6.2 CD: freeglut-2.6.0-1.el6.i686.rpm, freeglut-2.6.0-1.el6.x86_64.rpm, 
#                 freeglut-devel-2.6.0-1.el6.x86_64.rpm, mesa-libGL-devel-7.11-3.el6.x86_64.rpm
#                 mesa-libGLU-devel-7.11-3.el6.x86_64.rpm, libXfixes-devel-4.0.4-1.el6.x86_64.rpm
#                 libXdamage-devel-1.1.2-1.el6.x86_64.rpm, libXxf86vm-devel-1.1.0-1.el6.x86_64.rpm
#                 libdrm-devel-2.4.25-2.el6.x86_64.rpm
#
#
# In .bashrc or .bash_profile, add:
# export PATH=/usr/local/cuda/bin:/usr/local/cuda_sdk/C/bin/linux/release:$PATH
# export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda_sdk/C/lib:$$LD_LIBRARY_PATH

make -C /usr/local/cuda_sdk/C
</syntaxhighlight>

===== Installing driver on CentOS 6.2 =====

There didn't appear to be a kernel-source package in the CentOS repos. Installed kernel-devel and ran the installer with
<syntaxhighlight>
sh ./devdriver_4.0_linux_64_270.41.19.run --kernel-source-path /usr/src/kernels/kernel version
</syntaxhighlight>