Difference between revisions of "Solarflare SFN8522"

Revision as of 09:53, 20 June 2016

Onload Install

System requierments

Centos with full updates
Development tools

Installation

scripts/onload_install
onload_tool reload

Solar Flare Tests

Pre requisites

centos uptodate
Development tools

two servers connected by direct cable
Solarflare interfaces configured and pingbale
OpenOnload, netperf and sfnettest installed

netperf install

netperf can be downloaded from http://www.netperf.org/netperf/

tar -zxvf netperf-<version>.tar.gz
cd netperf-<version>
./configure
make install

sfnettest install

sfnettest can be downloaded from http://www.openonload.org

tar -zxvf sfnettest-<version>.tgz
cd sfnettest-<version>/src
make

pre test Config

Isolate the CPUs from, SMP balancing by adding teh following to the kernel line in boot.conf:

isolcpus=<comma separated cpu list>

Stop the services:

service cpuspeed stop
ice irqbalance stop
service iptables stop
ethtool -C eth<N> rx-usecs 0 adaptive-rx off

Check the BIOS settings so make sure that power saving is disabled

Steer the Interrupts to a CPU core used for the benchmark:

# cat /proc/interrupts | grep eth2
33: 0 0 0 0 IR-PCI-MSI-edge eth2-0
34: 0 0 0 0 IR-PCI-MSI-edge eth2-1

# echo 1 > /proc/irq/33/smp_affinity
# echo 2 > /proc/irq/34/smp_affinity

Latency Tests

Run sfnt-pingpong on both servers:

[root@system-1]# onload --profile=latency taskset -c 3 ./sfnt-pingpong
[root@system-2]# onload --profile=latency taskset -c 3 ./sfnt-pingpong \
--affinity "3;3" udp <system1-ip>

Latency Results

	size	mean	min	median	max	%ile	stddev	iter
	0	1675	1553	1668	923157	1760	977	890000
	1	1674	1543	1668	9182	1754	43	891000
	2	1674	1549	1668	8899	1756	41	891000
	4	1673	1539	1667	4036	1752	40	892000
	8	1673	1554	1667	11701	1751	40	892000
	16	1670	1531	1665	3991	1753	36	893000
	32	1704	1576	1699	4001	1785	39	875000
	64	1732	1641	1727	11127	1810	37	861000
	128	1841	1734	1837	4406	1936	44	811000
	256	2026	1902	2023	4446	2126	48	737000
	512	2264	2143	2260	4819	2365	48	660000
	1024	2881	2769	2876	5350	2992	56	519000
	1472	3439	3324	3433	5993	3557	57	435000

Netperf Tests

To accelerate an application with the Onload user-level transport, simply invoke the application on the command line as normal, prepended with "onload".

ie. Instead of:

  netperf -t TCP_RR -H myserver

do this:

  onload netperf -t TCP_RR -H myserver

oo:netperf[3420]: Using OpenOnload 201604-8000preview Copyright 2006-2016 Solarflare Communications, 2002-2005 Level 5 Networks [0]
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.20 () port 0 AF_INET : first burst 0
Local /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate         
bytes  Bytes  bytes    bytes   secs.    per sec   

16384  87380  1        1       10.00    67595.62   
16384  87380

and tuned for best latency:

  onload -p latency netperf -t TCP_RR -H myserver

oo:netperf[3494]: Using OpenOnload 201604-8000preview Copyright 2006-2016 Solarflare Communications, 2002-2005 Level 5 Networks [5]
MIGRATED TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 192.168.10.20 () port 0 AF_INET : first burst 0
Local /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate         
bytes  Bytes  bytes    bytes   secs.    per sec   

16384  87380  1        1       10.00    78216.20   
16384  87380

sqmr