Intel Build

Built using:

OpenMPI 1.4.3 (source /usr/mpi/intel/openmpi-1.4.3/bin/mpivars.sh ; built using intel composer 2013)
Intel compilers (source /share/apps/intel/composer_xe_2013.0.079/bin/compilervars.sh intel64)

Useful background on lammps:

Download source

Pull latest version from: http://lammps.sandia.gov/tars

  wget http://lammps.sandia.gov/tars/lammps.tar.gz
  tar zxvf lammps.tar.gz
  cd lammps/src/MAKE

Build MKL FFTW

  cd /share/apps/intel/composer_xe_2013.0.079/mkl/interfaces/fftw2xc
  make libintel64 compiler=intel

Edit the Makefile

Using openmpi for this build so edit Makefile.openmpi

CC =            icc
CCFLAGS =       -O2 -fno-exceptions -fno-rtti -ansi-alias -march=core2 -mtune=core2 -pc64 -no-prec-div -no-prec-sqrt -unroll
DEPFLAGS =      -M

LINK =          mpicc
LINKFLAGS =     -O2 -L/share/apps/intel/composer_xe_2013.0.079/mkl/lib/intel64/
LIB =           -L/share/apps/intel/composer_xe_2012.0.079/mkl/lib/intel64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lstdc++

FFT_INC =       -DFFT_FFTW -I/share/apps/intel/composer_xe_2013.0.079/mkl/include/fftw
FFT_PATH =
FFT_LIB =       /share/apps/intel/composer_xe_2013.0.079/mkl/lib/intel64/libfftw2xc_intel.a

MPI_INC =       -I/usr/mpi/intel/openmpi-1.4.3/include -DMPICH_IGNORE_CXX_SEEK
MPI_PATH =      -L/usr/mpi/intel/openmpi-1.4.3/lib64
MPI_LIB =

# From the original build flags
SHFLAGS =       -fPIC
SIZE =          size

Build

  cd ../ (to the src directory)
  make openmpi
  # this will create a file: lammps-5Oct12/src/lmp_openmpi

Run the tests

  #input file provided by customer
  cd ~/lammps-input
  mpirun -np 16 ../lammps-5Oct12/src/lmp_openmpi -i in.bmim_bf4

From the HPC Advisory council, here are some run time flags suggested for openmpi

  mpirun -np 168 -hostfile ~/hostfile-ompi.14 -mca mpi_paffinity_alone 1 \
      -mca btl self,sm,openib ~/lammps-30Aug10-gnu-openmpi-1.4.2/src/lmp_maia_gnu_openmpi \ 
      -in in.rhodo.scaled  -var x 4 -var y 6 -var z 7

GCC Build

Compilation using openmpi/gcc

Unpack the tar file
Navigate into src folder
Navigate into MAKE folder
Edit Makefile.openmpi

Remove the flags for FFTW libaray, save and exit

Naviagate to parent folder and type

make openmpi
cp lmp_openmpi ../bench
mpirun -np 4 lmp_openmpi < in.lj

Compilation using g++

Navigate into MAKE folder Edit Makefile.g++

Change compiler and linker to mpic++ as below:

# ---------------------------------------------------------------------
# compiler/linker settings
# specify flags and libraries needed for your compiler

CC =            mpic++
CCFLAGS =       -g -O # -Wunused
SHFLAGS =       -fPIC
DEPFLAGS =      -M

LINK =          mpic++
LINKFLAGS =     -g -O
LIB =
SIZE =          size

ARCHIVE =       ar
ARFLAGS =       -rc
SHLIBFLAGS =    -shared

Remove the MPI flags

Compile as for openmpi

GPU Build

http://lammps.sandia.gov/doc/Section_accelerate.html#acc_6

compile the cuda library
add to installation
compile lammps

Edit makefile.common to point to the cuda libraries

cd lib/cuda
make

check Makefile.lammps matches to path to the cuda libraries

cd ../../src
make yes-user-cuda
make linux

This will produce an executable with the same name as the above builds. I would suggest renaming the binary to avaoid confusion.

There are some example input files provided. The benchmark is run in the same way as before. The only chnages that need to be made are to the input files. You need to add the cuda package, and use the cuda kspace styles.

Ful details can be found on the lammps website.

# Rhodopsin model

package         cuda gpu/node 2

variable        x index 2
variable        y index 2
variable        z index 2

units           real
neigh_modify    delay 5 every 1

atom_style      full
bond_style      harmonic
angle_style     charmm
dihedral_style  charmm
improper_style  harmonic
pair_style      lj/charmm/coul/long 8.0 10.0
pair_modify     mix arithmetic
kspace_style    pppm/cuda 1e-4

read_data       data.rhodo

replicate       $x $y $z

fix             1 all shake 0.0001 5 0 m 1.0 a 232
fix             2 all npt temp 300.0 300.0 100.0 &
                z 0.0 0.0 1000.0 mtk no pchain 0 tchain 1
special_bonds   charmm

thermo          100
thermo_style    multi
timestep        2.0

run             1000

Benchmarking: LAMMPS

Contents