http://wiki.define-technology.com/mediawiki-1.35.0/index.php?action=history&feed=atom&title=Benchmarking%3A_MPI_Message_Rates%3A_SQMR Benchmarking: MPI Message Rates: SQMR - Revision history 2026-05-04T18:52:50Z Revision history for this page on the wiki MediaWiki 1.35.0 http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=Benchmarking:_MPI_Message_Rates:_SQMR&diff=1123&oldid=prev David: Created page with "== Download == * Available from: https://asc.llnl.gov/sequoia/benchmarks/#phloem * Also part of the cbench (http://sourceforge.net/apps/trac/cbench) <syntaxhighlight> wget https://asc.ll..." 2012-10-12T15:19:06Z <p>Created page with &quot;== Download == * Available from: https://asc.llnl.gov/sequoia/benchmarks/#phloem * Also part of the cbench (http://sourceforge.net/apps/trac/cbench) &lt;syntaxhighlight&gt; wget https://asc.ll...&quot;</p> <p><b>New page</b></p><div>== Download ==<br /> * Available from: https://asc.llnl.gov/sequoia/benchmarks/#phloem<br /> * Also part of the cbench (http://sourceforge.net/apps/trac/cbench)<br /> &lt;syntaxhighlight&gt;<br /> wget https://asc.llnl.gov/sequoia/benchmarks/phloem_v1.0.tgz<br /> &lt;/syntaxhighlight&gt;<br /> <br /> == Build ==<br /> * tar, extract etc and run make (make sure mpicc is setup in PATH)<br /> &lt;syntaxhighlight&gt;<br /> [viglen@fp2-hn phloem-1.0.0]$ which mpicc <br /> /opt/platform_mpi/v8.1/bin/mpicc<br /> [viglen@fp2-hn phloem-1.0.0]$ make <br /> ================================================================================<br /> Building benchmark presta-1.0.0<br /> ================================================================================<br /> make[1]: Entering directory `/home/viglen/scratch/phloem-1.0.0/presta-1.0.0&#039;<br /> mpicc -DPRINT_ENV -O2 -g -c com.c<br /> mpicc -DPRINT_ENV -O2 -g -c util.c<br /> mpicc -DPRINT_ENV -O2 -g -o com com.o util.o <br /> make[1]: Leaving directory `/home/viglen/scratch/phloem-1.0.0/presta-1.0.0&#039;<br /> ================================================================================<br /> Building benchmark sqmr-1.0.0<br /> ================================================================================<br /> make[1]: Entering directory `/home/viglen/scratch/phloem-1.0.0/sqmr-1.0.0&#039;<br /> mpicc -DPRINT_ENV -O2 -g -c sqmr.c<br /> mpicc -o sqmr sqmr.o ../presta*/util.o<br /> make[1]: Leaving directory `/home/viglen/scratch/phloem-1.0.0/sqmr-1.0.0&#039;<br /> ================================================================================<br /> Building benchmark mpiBench-1.0.0<br /> ================================================================================<br /> make[1]: Entering directory `/home/viglen/scratch/phloem-1.0.0/mpiBench-1.0.0&#039;<br /> mpicc -DPRINT_ENV -O2 -g -DVERS=\&quot;1.0.0\&quot; -o mpiBench_Allreduce mpiBench_Allreduce.c ../presta*/util.o<br /> mpicc -DPRINT_ENV -O2 -g -DVERS=\&quot;1.0.0\&quot; -o mpiBench_Barrier mpiBench_Barrier.c ../presta*/util.o<br /> mpicc -DPRINT_ENV -O2 -g -DVERS=\&quot;1.0.0\&quot; -o mpiBench_Bcast mpiBench_Bcast.c ../presta*/util.o<br /> make[1]: Leaving directory `/home/viglen/scratch/phloem-1.0.0/mpiBench-1.0.0&#039;<br /> ================================================================================<br /> Building benchmark linktest-1.0.0<br /> ================================================================================<br /> make[1]: Entering directory `/home/viglen/scratch/phloem-1.0.0/linktest-1.0.0&#039;<br /> mkdir -p linux<br /> mpicc -c bmtime.c -o linux/bmtime.o -DPRINT_ENV -O2 -g <br /> mpicc -c nodelib-linux.c -o linux/nodelib.o -DPRINT_ENV -O2 -g <br /> mpicc -c linktest.c -o linux/linktest.o -DPRINT_ENV -O2 -g <br /> mpicc -o linux/linktest linux/bmtime.o linux/nodelib.o linux/linktest.o ../presta*/util.o<br /> Compiled linktest with ARCH=linux using nodelib-linux.c<br /> make[1]: Leaving directory `/home/viglen/scratch/phloem-1.0.0/linktest-1.0.0&#039;<br /> ================================================================================<br /> Building benchmark torustest-1.0.0<br /> ================================================================================<br /> make[1]: Entering directory `/home/viglen/scratch/phloem-1.0.0/torustest-1.0.0&#039;<br /> mkdir -p linux<br /> mpicc -c bmtime.c -o linux/bmtime.o -DPRINT_ENV -O2 -g <br /> mpicc -c torustest.c -o linux/torustest.o -DPRINT_ENV -O2 -g <br /> mpicc -o linux/torustest linux/bmtime.o linux/torustest.o -lm ../presta*/util.o<br /> mpicc -o linux/generate -DPRINT_ENV -O2 -g generate.c<br /> Compiled torustest with ARCH=linux using nodelib-linux.c<br /> make[1]: Leaving directory `/home/viglen/scratch/phloem-1.0.0/torustest-1.0.0&#039;<br /> ================================================================================<br /> Building benchmark mpiGraph-1.0.0<br /> ================================================================================<br /> make[1]: Entering directory `/home/viglen/scratch/phloem-1.0.0/mpiGraph-1.0.0&#039;<br /> rm -rf mpiGraph.o mpiGraph mpiGraph.out mpiGraph.tgz<br /> mpicc -DPRINT_ENV -O2 -g -o mpiGraph mpiGraph.c ../presta*/util.o<br /> make[1]: Leaving directory `/home/viglen/scratch/phloem-1.0.0/mpiGraph-1.0.0&#039;<br /> [viglen@fp2-hn phloem-1.0.0]$ ls<br /> linktest-1.0.0 Makefile.inc mpiGraph-1.0.0 presta-1.0.0 README.sow sqmr-1.0.0<br /> Makefile mpiBench-1.0.0 PhloemMPIBenchmarks_summary_v1.0.0.pdf README run_script torustest-1.0.0<br /> &lt;/syntaxhighlight&gt;<br /> * The build process will create a folder called sqmr-1.0.0. (Sequioa message rate)<br /> <br /> <br /> == Run ==<br /> * Run on Qlogic IB (include -PSM)<br /> &lt;syntaxhighlight&gt;<br /> [viglen@fp2-1 sqmr-1.0.0]$ mpirun -np 2 -PSM -hostlist &quot;fp2-1 fp2-2&quot; ./sqmr<br /> ################################################################################<br /> # SQMR v1.0.0 - MPI maximal message rate benchmark<br /> # Run at 06/27/11 05:17:43, with rank 0 on fp2-1<br /> #<br /> # MPI tasks per node : 1<br /> # Neighbor tasks : 1<br /> # Iterations per message size : 4096<br /> # Send/Recv operations per iteration : 1<br /> #<br /> # average max min<br /> # msgsize iters time msgs/sec MiB/sec msgs/sec MiB/sec msgs/sec MiB/sec<br /> 0 4096 0.01 870461.79 0.00 870461.79 0.00 870461.79 0.00<br /> 1 3277 0.01 890722.20 0.85 890722.20 0.85 890722.20 0.85<br /> 2 2622 0.01 890627.23 1.70 890627.23 1.70 890627.23 1.70<br /> 4 2098 0.00 895046.51 3.41 895046.51 3.41 895046.51 3.41<br /> 8 1679 0.00 884425.30 6.75 884425.30 6.75 884425.30 6.75<br /> 16 1344 0.00 740900.91 11.31 740900.91 11.31 740900.91 11.31<br /> 32 1076 0.00 742831.22 22.67 742831.22 22.67 742831.22 22.67<br /> 64 861 0.00 736172.82 44.93 736172.82 44.93 736172.82 44.93<br /> 128 689 0.00 715050.22 87.29 715050.22 87.29 715050.22 87.29<br /> 256 552 0.00 669923.56 163.56 669923.56 163.56 669923.56 163.56<br /> 512 442 0.00 576724.96 281.60 576724.96 281.60 576724.96 281.60<br /> 1024 354 0.00 477038.91 465.86 477038.91 465.86 477038.91 465.86<br /> 2048 284 0.00 374467.88 731.38 374467.88 731.38 374467.88 731.38<br /> 4096 228 0.00 238241.48 930.63 238241.48 930.63 238241.48 930.63<br /> 8192 183 0.00 108473.38 847.45 108473.38 847.45 108473.38 847.45<br /> 16384 147 0.00 71270.68 1113.60 71270.68 1113.60 71270.68 1113.60<br /> 32768 118 0.01 45986.33 1437.07 45986.33 1437.07 45986.33 1437.07<br /> 65536 95 0.01 30769.03 1923.06 30769.03 1923.06 30769.03 1923.06<br /> 131072 76 0.01 18377.51 2297.19 18377.51 2297.19 18377.51 2297.19<br /> 262144 61 0.01 11487.63 2871.91 11487.63 2871.91 11487.63 2871.91<br /> 524288 49 0.01 6572.36 3286.18 6572.36 3286.18 6572.36 3286.18<br /> 1048576 40 0.02 3426.86 3426.86 3426.86 3426.86 3426.86 3426.86<br /> 2097152 32 0.04 1817.88 3635.76 1817.88 3635.76 1817.88 3635.76<br /> 4194304 26 0.06 943.38 3773.52 943.38 3773.52 943.38 3773.52<br /> &lt;/syntaxhighlight&gt;<br /> * Run on ethernet<br /> &lt;syntaxhighlight&gt;<br /> [viglen@fp2-1 sqmr-1.0.0]$ mpirun -np 2 -TCP -hostlist &quot;fp2-1 fp2-2&quot; ./sqmr<br /> ################################################################################<br /> # SQMR v1.0.0 - MPI maximal message rate benchmark<br /> # Run at 06/27/11 05:19:31, with rank 0 on fp2-1<br /> #<br /> # MPI tasks per node : 1<br /> # Neighbor tasks : 1<br /> # Iterations per message size : 4096<br /> # Send/Recv operations per iteration : 1<br /> #<br /> # average max min<br /> # msgsize iters time msgs/sec MiB/sec msgs/sec MiB/sec msgs/sec MiB/sec<br /> 0 4096 0.10 83743.78 0.00 83743.78 0.00 83743.78 0.00<br /> 1 3277 0.08 83218.96 0.08 83218.96 0.08 83218.96 0.08<br /> 2 2622 0.06 83505.81 0.16 83505.81 0.16 83505.81 0.16<br /> 4 2098 0.05 82844.40 0.32 82844.40 0.32 82844.40 0.32<br /> 8 1679 0.04 80727.19 0.62 80727.19 0.62 80727.19 0.62<br /> 16 1344 0.03 80052.04 1.22 80052.04 1.22 80052.04 1.22<br /> 32 1076 0.03 78692.11 2.40 78692.11 2.40 78692.11 2.40<br /> 64 861 0.02 75815.79 4.63 75815.79 4.63 75815.79 4.63<br /> 128 689 0.02 60134.95 7.34 60134.95 7.34 60134.95 7.34<br /> 256 552 0.02 58286.49 14.23 58286.49 14.23 58286.49 14.23<br /> 512 442 0.02 36594.24 17.87 36594.24 17.87 36594.24 17.87<br /> 1024 354 0.03 21331.11 20.83 21331.11 20.83 21331.11 20.83<br /> 2048 284 0.05 11034.27 21.55 11034.27 21.55 11034.27 21.55<br /> 4096 228 0.05 8357.93 32.65 8357.93 32.65 8357.93 32.65<br /> 8192 183 0.05 8074.67 63.08 8074.67 63.08 8074.67 63.08<br /> 16384 147 0.04 6941.83 108.47 6941.83 108.47 6941.83 108.47<br /> 32768 118 0.08 3031.12 94.72 3031.12 94.72 3031.12 94.72<br /> 65536 95 0.09 2035.05 127.19 2035.05 127.19 2035.05 127.19<br /> 131072 76 0.12 1248.99 156.12 1248.99 156.12 1248.99 156.12<br /> 262144 61 0.17 725.46 181.37 725.46 181.37 725.46 181.37<br /> 524288 49 0.25 398.37 199.18 398.37 199.18 398.37 199.18<br /> 1048576 40 0.38 208.50 208.50 208.50 208.50 208.50 208.50<br /> 2097152 32 0.59 107.92 215.83 107.92 215.83 107.92 215.83<br /> 4194304 26 0.95 55.01 220.05 55.01 220.05 55.01 220.05<br /> &lt;/syntaxhighlight&gt;<br /> <br /> * As you can see above, a factor of 10x improvement running on QDR over 1GB eth</div> David