Benchmarking: LAMMPS

Intel Build

Built using:

• OpenMPI 1.4.3 (source /usr/mpi/intel/openmpi-1.4.3/bin/mpivars.sh ; built using intel composer 2013)
• Intel compilers (source /share/apps/intel/composer_xe_2013.0.079/bin/compilervars.sh intel64)

Useful background on lammps:

• http://www.hpcadvisorycouncil.com/pdf/LAMMPS_Best_Practice.pdf
• http://www.hpcadvisorycouncil.com/pdf/LAMMPS_Analysis_and_Profiling_Intel.pdf

Download source

Pull latest version from: http://lammps.sandia.gov/tars

  wget http://lammps.sandia.gov/tars/lammps.tar.gz
  tar zxvf lammps.tar.gz
  cd lammps/src/MAKE

Build MKL FFTW

  cd /share/apps/intel/composer_xe_2013.0.079/mkl/interfaces/fftw2xc
  make libintel64 compiler=intel

Edit the Makefile

Using openmpi for this build so edit Makefile.openmpi

CC =            icc
CCFLAGS =       -O2 -fno-exceptions -fno-rtti -ansi-alias -march=core2 -mtune=core2 -pc64 -no-prec-div -no-prec-sqrt -unroll
DEPFLAGS =      -M

LINK =          mpicc
LINKFLAGS =     -O2 -L/share/apps/intel/composer_xe_2013.0.079/mkl/lib/intel64/
LIB =           -L/share/apps/intel/composer_xe_2012.0.079/mkl/lib/intel64 -lmkl_lapack95_lp64 -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -lstdc++

FFT_INC =       -DFFT_FFTW -I/share/apps/intel/composer_xe_2013.0.079/mkl/include/fftw
FFT_PATH =
FFT_LIB =       /share/apps/intel/composer_xe_2013.0.079/mkl/lib/intel64/libfftw2xc_intel.a

MPI_INC =       -I/usr/mpi/intel/openmpi-1.4.3/include -DMPICH_IGNORE_CXX_SEEK
MPI_PATH =      -L/usr/mpi/intel/openmpi-1.4.3/lib64
MPI_LIB =

# From the original build flags
SHFLAGS =       -fPIC
SIZE =          size

Build

  cd ../ (to the src directory)
  make openmpi
  # this will create a file: lammps-5Oct12/src/lmp_openmpi

Run the tests

  #input file provided by customer
  cd ~/lammps-input
  mpirun -np 16 ../lammps-5Oct12/src/lmp_openmpi -i in.bmim_bf4

From the HPC Advisory council, here are some run time flags suggested for openmpi

  mpirun -np 168 -hostfile ~/hostfile-ompi.14 -mca mpi_paffinity_alone 1 \
      -mca btl self,sm,openib ~/lammps-30Aug10-gnu-openmpi-1.4.2/src/lmp_maia_gnu_openmpi \ 
      -in in.rhodo.scaled  -var x 4 -var y 6 -var z 7

GCC Build

Compilation using openmpi/gcc

1. Unpack the tar file
2. Navigate into src folder
3. Navigate into MAKE folder
4. Edit Makefile.openmpi

Remove the flags for FFTW libaray, save and exit

Naviagate to parent folder and type

make openmpi
cp lmp_openmpi ../bench
mpirun -np 4 lmp_openmpi < in.lj

Compilation using g++

Navigate into MAKE folder Edit Makefile.g++

Change compiler and linker to mpic++ as below:

# ---------------------------------------------------------------------
# compiler/linker settings
# specify flags and libraries needed for your compiler

CC =            mpic++
CCFLAGS =       -g -O # -Wunused
SHFLAGS =       -fPIC
DEPFLAGS =      -M

LINK =          mpic++
LINKFLAGS =     -g -O
LIB =
SIZE =          size

ARCHIVE =       ar
ARFLAGS =       -rc
SHLIBFLAGS =    -shared

Remove the MPI flags

Compile as for openmpi

GPU Build

1. compile the GPU library
2. add to installation
3. compile lammps

cd /lib/gpu
make -f Makefile.linux

edit Makefile.lammps to match to path to the cuda libraries

cd ../../src
make yes-gpu
make linux