Benchmarking: Stream (Memory Bandwidth) - Revision history

Damian: /* Dual Skylake 6132 */

2018-03-14T10:17:54Z

Dual Skylake 6132

Damian at 22:53, 13 March 2018

2018-03-13T22:53:31Z

David: /* Haswell E5-2650v3 DDR4-2133Mhz */

2014-09-18T15:31:29Z

Haswell E5-2650v3 DDR4-2133Mhz

David: /* Haswell DDR4 1866Mhz E5-2640v3 */

2014-09-18T15:03:57Z

Haswell DDR4 1866Mhz E5-2640v3

David: /* Quad Socket E5-4640v2 */

2014-09-18T11:26:50Z

Quad Socket E5-4640v2

David at 11:11, 18 September 2014

2014-09-18T11:11:12Z

David: /* Single Calxeda ECX-2000 */

2014-09-18T10:47:38Z

Single Calxeda ECX-2000

Jon: /* Results */

2013-12-11T09:52:12Z

Results

David at 20:45, 25 September 2013

2013-09-25T20:45:03Z

David at 20:42, 25 September 2013

2013-09-25T20:42:55Z

@@ Line 337: / Line 337: @@
 == Dual Skylake 6132 ==
-#only logical cores used for test (2x14=28cores)
+# only logical cores used for test (2x14=28cores)
 <syntaxhighlight>
-#12 sticks of 16gb 2666Mhz
+# 12 sticks of 16gb 2666Mhz
-#/opt/intel/bin/icc -O3 -qopenmp -parallel -AVX -DSTREAM_ARRAY_SIZE=800000000 -mcmodel=medium stream.c -o stream_icc_AVX
+# /opt/intel/bin/icc -O3 -qopenmp -parallel -AVX -DSTREAM_ARRAY_SIZE=800000000 -mcmodel=medium stream.c -o stream_icc_AVX
-#for j in 28; do for i in {1..100}; do OMP_NUM_THREADS=$j ./stream_icc_AVX | grep "Copy\|Scale\|Add\|Triad\|counted" | tee -a skylake_6132_12x16gb_stream_icc_AVX_28core.txt ;done ;done
+# for j in 28; do for i in {1..100}; do OMP_NUM_THREADS=$j ./stream_icc_AVX | grep "Copy\|Scale\|Add\|Triad\|counted" | tee -a skylake_6132_12x16gb_stream_icc_AVX_28core.txt ;done ;done
-#grep the best results: cat skylake_6132_12x16gb_stream_icc_AVX_28core.txt | sort -k 4 | grep -i copy
+# grep the best results: cat skylake_6132_12x16gb_stream_icc_AVX_28core.txt | sort -k 4 | grep -i copy
 Copy:          140025.6     0.092183     0.091412     0.092748
 Scale:         141014.3     0.091679     0.090771     0.092523
@@ Line 348: / Line 348: @@
 Triad:         141462.5     0.136485     0.135725     0.137369
-#6 sticks of 32gb 2666Mhz
+# 6 sticks of 32gb 2666Mhz
-#/opt/intel/bin/icc -O3 -qopenmp -parallel -AVX -DSTREAM_ARRAY_SIZE=800000000 -mcmodel=medium stream.c -o stream_icc_AVX
+# /opt/intel/bin/icc -O3 -qopenmp -parallel -AVX -DSTREAM_ARRAY_SIZE=800000000 -mcmodel=medium stream.c -o stream_icc_AVX
-#for j in 28; do for i in {1..100}; do OMP_NUM_THREADS=$j ./stream_icc_AVX | grep "Copy\|Scale\|Add\|Triad\|counted" | tee -a skylake_6132_6x32gb_stream_icc_AVX_28core.txt ;done ;done
+# for j in 28; do for i in {1..100}; do OMP_NUM_THREADS=$j ./stream_icc_AVX | grep "Copy\|Scale\|Add\|Triad\|counted" | tee -a skylake_6132_6x32gb_stream_icc_AVX_28core.txt ;done ;done
 #grep the best results: cat skylake_6132_6x32gb_stream_icc_AVX_28core.txt | sort -k 4 | grep -i copy
 Copy:           94531.2     0.136154     0.135405     0.137249
@@ Line 356: / Line 356: @@
 Add:            97745.7     0.198002     0.196428     0.199466
 Triad:         100247.4     0.192429     0.191526     0.192869
+== Dual Skylake 5120 ==
+#only logical cores used for test (2x14=28cores)
 <syntaxhighlight>
 </syntaxhighlight>

@@ Line 333: / Line 333: @@
    Add:            63344.1     0.115197     0.113665     0.116755
    Triad:          64233.5     0.112643     0.112091     0.114209
 </syntaxhighlight>

← Older revision		Revision as of 15:31, 18 September 2014
Line 224:		Line 224:

	=== Haswell E5-2650v3 DDR4-2133Mhz ===		=== Haswell E5-2650v3 DDR4-2133Mhz ===
		+	<syntaxhighlight>
		+	# HT on
		+	[root@e5-2650v3-nodee stream]# icc -O3 -openmp -DSTREAM_ARRAY_SIZE=4500000000 -mcmodel=medium stream.c -o stream_icc_dp
		+	[root@e5-2650v3-nodee stream]# OMP_NUM_THREADS=20 KMP_AFFINITY=scatter ./stream_icc_dp
		+	-------------------------------------------------------------
		+	STREAM version $Revision: 5.10 $
		+	-------------------------------------------------------------
		+	This system uses 8 bytes per array element.
		+	-------------------------------------------------------------
		+	Array size = 4500000000 (elements), Offset = 0 (elements)
		+	Memory per array = 34332.3 MiB (= 33.5 GiB).
		+	Total memory required = 102996.8 MiB (= 100.6 GiB).
		+	Each kernel will be executed 10 times.
		+	The best time for each kernel (excluding the first iteration)
		+	will be used to compute the reported bandwidth.
		+	-------------------------------------------------------------
		+	Number of Threads requested = 20
		+	Number of Threads counted = 20
		+	-------------------------------------------------------------
		+	Your clock granularity/precision appears to be 1 microseconds.
		+	Each test below will take on the order of 714150 microseconds.
		+	(= 714150 clock ticks)
		+	Increase the size of the arrays if this shows that
		+	you are not getting at least 20 clock ticks per test.
		+	-------------------------------------------------------------
		+	WARNING -- The above is only a rough guideline.
		+	For best results, please be sure you know the
		+	precision of your system timer.
		+	-------------------------------------------------------------
		+	Function Best Rate MB/s Avg time Min time Max time
		+	Copy: 107295.5 0.671414 0.671044 0.672320
		+	Scale: 107487.7 0.670575 0.669844 0.671884
		+	Add: 110500.1 0.978171 0.977375 0.981087
		+	Triad: 111482.3 0.969603 0.968764 0.972643
		+	-------------------------------------------------------------
		+	Solution Validates: avg error less than 1.000000e-13 on all three arrays
		+	-------------------------------------------------------------
		+
		+	</syntaxhighlight>

	=== Quad Socket E5-4640v2 ===		=== Quad Socket E5-4640v2 ===

@@ Line 183: / Line 183: @@
 </syntaxhighlight>
-=== Haswell DDR4 1866Mhz E5-2640v3 ===
+=== Haswell E5-2640v3 DDR4-1866Mhz  ===
 <syntaxhighlight>
 # HT Disabled / service cpuspeed stop
@@ Line 222: / Line 222: @@
 -------------------------------------------------------------
 </syntaxhighlight>
 === Quad Socket E5-4640v2 ===

@@ Line 224: / Line 224: @@
 === Quad Socket E5-4640v2 ===
-* With 128GB RAM
+* With 128GB RAM, HT enabled but running half the cores (80 cores total)
 == Observations ==

@@ Line 181: / Line 181: @@
 Add:             4051.3     0.059573     0.059240     0.059767
 Triad:           3847.4     0.062560     0.062380     0.062768
 </syntaxhighlight>

@@ Line 155: / Line 155: @@
    Add:           213480.2     0.171661     0.168634     0.180814
    Triad:         215921.8     0.170627     0.166727     0.177991
 </syntaxhighlight>

← Older revision		Revision as of 20:45, 25 September 2013
Line 24:		Line 24:

	The fourth benchmark in Stream, the Triad benchmark, allows chained or overlapped or fused, multiple-add operations. It builds on the Sum benchmark by adding an arithmetic operation to one of the fetched array values. Given that fused multiple-add operations (FMA) are an important operation in many basic computations, such as dot products, matrix multiplication, polynomial evaluations, Newton’s method for evaluation functions, and many DSP operations, this benchmark can be directly associated with application performance. The FMA operation has its own instruction set now and is usually done in hardware. Consequently, feeding such hardware operations with data can be extremely important – hence, the usefulness of the Triad memory bandwidth benchmark.		The fourth benchmark in Stream, the Triad benchmark, allows chained or overlapped or fused, multiple-add operations. It builds on the Sum benchmark by adding an arithmetic operation to one of the fetched array values. Given that fused multiple-add operations (FMA) are an important operation in many basic computations, such as dot products, matrix multiplication, polynomial evaluations, Newton’s method for evaluation functions, and many DSP operations, this benchmark can be directly associated with application performance. The FMA operation has its own instruction set now and is usually done in hardware. Consequently, feeding such hardware operations with data can be extremely important – hence, the usefulness of the Triad memory bandwidth benchmark.
		+
		+
		+	There are two variables or definitions in the code that you should pay attention to. The first is STREAM_ARRAY_SIZE. This is the number of array elements used to run the benchmarks. In the current version, it is set to 10,000,000, which the code states should be good enough for caches up to 20MB. The Stream FAQ recommends you use a problem size such that each array is four times the sum of the caches (L1, L2, and L3). You can either change the code to reflect the array sizes you want, or you can set the variable when compiling the code.
		+
		+
		+
		+	The second variable you might want to change is NTIMES, the number of times each benchmark is run. By default, Stream reports the "best" result for any iteration after the first; therefore, be sure always to set NTIMES at least to 2 (10 is the default). This variable can also be set during compilation without changing the code.

@@ Line 4: / Line 4: @@
 Table 1: Stream Benchmarks
-Name	 Kernel	           Bytes/Iteration	 FLOPS/Iteration
+Name	 Kernel	              Bytes/Iteration	 FLOPS/Iteration
 COPY	a(i) = b(i)	           16	 0
 SCALE	a(i) = q*b(i)	           16	 1
-SUM	a(i) = b(i) + c(i)	    24	 1
+SUM	a(i) = b(i) + c(i)	   24	 1
 TRIAD	a(i) = b(i) + q*c(i)	   24	 2
   </syntaxhighlight>