Benchmarking: Star-CCM+ (CD-Adapco)

Install Star-CCM+

tar zxvf STAR-CCM+9.04.011_01_linux-x86_64.tar.gz 
cd starccm+_9.04.011/
./STAR-CCM+9.04.011_01_linux-x86_64-2.5_gnu4.6.bin

It will ask you a couple of question like installation directory etc. Pretty straight forward.

License

Uses flexlm. Managed to persuade CD adapco for a temp license which was granted against their external flexlm server

[david@haswell5 StarCCM]$ cat flexlmrc 
CDLMD_LICENSE_FILE=1999@flex.cd-adapco.com
LM_PROJECT=MU5wchMhgrYL2mxDipZ+zw

Input files

Provided by end customer: Ask for a .sim file

[david@haswell5 StarCCM]$ pwd 
/ramdisk/dp/StarCCM
[david@haswell5 StarCCM]$ ls -lah
total 4.6G
drwxr-xr-x 2 david david  220 Oct  2 15:35 .
drwxrwxr-x 5 david david  160 Oct  2 14:12 ..
-rw-r--r-- 1 root  root    78 Oct  2 15:03 flexlmrc
-rw-rw-r-- 1 david david   48 Oct  2 15:13 nodes
-rw-r--r-- 1 david david  332 Sep 10 13:00 README
-rw-rw-r-- 1 david david 246K Oct  2 15:37 run-4-64.txt
-rwxrwxr-x 1 david david  675 Oct  2 15:12 run-star-cmm.sh
-rwxr-xr-x 1 root  root   744 Oct  2 15:03 star-bench.sh
-rwxr-xr-x 1 david david  744 Sep 10 12:14 wj.sh
-rw-rw-r-- 1 david david 2.5G Oct  2 15:35 wj-withXoverpipe@01000.sim # <-- these files get wrote every 1000 steps
-rw-r--r-- 1 david david 2.1G Jun  2 14:35 wj-withXoverpipe.sim # <-- this is this input file

Sample script they provided

[david@haswell5 StarCCM]$ cat star-bench.sh 
#!/bin/bash
### PBS ###
#PBS -S /bin/bash -j oe -k o -r n
#PBS -q cfdq 
#PBS -l select=3:ncpus=10:mpiprocs=10:mem=12gb
#PBS -l place=scatter:excl
export PATH=$PATH:/panfs/hpc/apps/starccm+/STAR-CCM+7.06.009-R8/star/bin/
export RAMFILES=1 
cd ${PBS_O_WORKDIR}

##export TMPDIR=dev/shm
#
# You must set SGI_MPI_HOME to point to your sgimpi # MPT 2.03 (or above) is recommended
#export SGI_MPI_HOME=/panfs/hpc/apps/sgi/mpt/mpt-2.03/
#export MPI_DSM_DISTRIBUTE
#export MPI_VERBOSE
####
(time starccm+ -np 30 -rsh ssh \
                 -batchsystem "pbs"  -batch \
                 -mppflags "-IBV -prot -cpu_bind=map_cpu=1,2,3,4,5,6,7,8,9,10,11,0" \
                ./wj-withXoverpipe.sim )>& out.txt # end of PBS script example
####				
exit 0

Script i ended up using

[david@haswell5 StarCCM]$ cat run-star-cmm.sh 
#!/bin/bash

# the version of Star-CCM we wish to use
CCMVER=9.04.011
export PATH=/ramdisk/dp/cd-adapco/STAR-CCM+9.04.011/star/bin/:$PATH

# define the port and license server
export CDLMD_LICENSE_FILE=1999@flex.cd-adapco.com
export LM_PROJECT=MU5wchMhgrYL2mxDipZ+zw

#
# OR use this flag to starccm+    -podkey MU5wchMhgrYL2mxDipZ+zw

RUNNAME=run-4-80



cd /ramdisk/dp/StarCCM

# 

# (time starccm+  `echo $PARAMS`     ./wj-withXoverpipe.sim )>& run1.txt  

time (starccm+  -power -time -batch -batch-report -np 64 -machinefile ./nodes -rsh /usr/bin/ssh -mpi platform -mpiflags "-IBV -prot " -cpubind bandwidth,v /ramdisk/dp/StarCCM/wj-withXoverpipe.sim  ) >& run-4-64.txt