David at 14:26, 18 July 2014

2014-07-18T14:26:01Z

David: Created page with "== Benchmark using Sockperf == === Check the PCI NIC locality == * Ensure that we are pinning the processes to the cores with the PCI controller [root@hft2 ~]# cat /..."

2014-07-18T14:25:00Z

Created page with "== Benchmark using Sockperf == === Check the PCI NIC locality == * Ensure that we are pinning the processes to the cores with the PCI controller <syntaxhighlight> [root@hft2 ~]# cat /..."

New page

== Benchmark using Sockperf ==

=== Check the PCI NIC locality ==

* Ensure that we are pinning the processes to the cores with the PCI controller

<syntaxhighlight>
[root@hft2 ~]# cat /sys/class/net/eth5/device/local_cpulist
8-15
</syntaxhighlight>

* Make sure the boot args specify <tt>isolcpus</tt> for the correct cores to isolated from the kernel scheduler.
<syntaxhighlight>
title CentOS (2.6.32-431.20.3.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-431.20.3.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=auto rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll pcie_aspm=performance mce=ignore_ce isolcpus=3,5,10,12
initrd /initramfs-2.6.32-431.20.3.el6.x86_64.img
</syntaxhighlight>

* Check this at run time with:
<syntaxhighlight>
[root@hft2 ~]# cat /proc/cmdline
ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=133M@0M rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet intel_idle.max_cstate=0 processor.max_cstate=0 idle=poll pcie_aspm=performance mce=ignore_ce isolcpus=3,5,10,12
</syntaxhighlight>

* Then pin the tasks on server/client to the correct cores (in this instance 12, as our range was 8-15, 12 just selected at random)

<syntaxhighlight>
# on the server side
[root@hft2 ~]# exasock taskset -c 12 sockperf sr
sockperf: == version #2.5.241 ==
sockperf: [SERVER] listen on:
[ 0] IP = 0.0.0.0 PORT = 11111 # UDP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: [tid 2827] using recvfrom() to block on socket(s)
</syntaxhighlight>

* Then on the client side
<syntaxhighlight>
# on the client side
[root@hft1 ~]# exasock taskset -c 12 sockperf pp -i 192.168.1.8 -t 5 -m 12
sockperf: == version #2.5.241 ==
sockperf[CLIENT] send on:sockperf: using recvfrom() to block on socket(s)

[ 0] IP = 192.168.1.8 PORT = 11111 # UDP
sockperf: Warmup stage (sending a few dummy messages)...
sockperf: Starting test...
sockperf: Test end (interrupted by timer)
sockperf: Test ended
sockperf: [Total Run] RunTime=5.100 sec; SentMessages=2437259; ReceivedMessages=2437258
sockperf: ========= Printing statistics for Server No: 0
sockperf: [Valid Duration] RunTime=5.000 sec; SentMessages=2391308; ReceivedMessages=2391308
sockperf: ====> avg-lat= 1.034 (std-dev=0.019)
sockperf: # dropped messages = 0; # duplicated messages = 0; # out-of-order messages = 0
sockperf: Summary: Latency is 1.034 usec
sockperf: Total 2391308 observations; each percentile contains 23913.08 observations
sockperf: ---> <MAX> observation = 11.844
sockperf: ---> percentile 99.99 = 1.501
sockperf: ---> percentile 99.90 = 1.377
sockperf: ---> percentile 99.50 = 1.067
sockperf: ---> percentile 99.00 = 1.053
sockperf: ---> percentile 95.00 = 1.045
sockperf: ---> percentile 90.00 = 1.041
sockperf: ---> percentile 75.00 = 1.037
sockperf: ---> percentile 50.00 = 1.033
sockperf: ---> percentile 25.00 = 1.029
sockperf: ---> <MIN> observation = 1.013
</syntaxhighlight>

@@ Line 1: / Line 1: @@
 == Benchmark using Sockperf ==
-=== Check the PCI NIC locality ==
+=== Check the PCI NIC locality ===
 * Ensure that we are pinning the processes to the cores with the PCI controller
@@ Line 10: / Line 10: @@
 </syntaxhighlight>
 * Make sure the boot args specify <tt>isolcpus</tt> for the correct cores to isolated from the kernel scheduler.
 <syntaxhighlight>
@@ Line 24: / Line 25: @@
 </syntaxhighlight>
 * Then pin the tasks on server/client to the correct cores (in this instance 12, as our range was 8-15, 12 just selected at random)

Exablaze: Setup and Benchmark ExaNIC - Revision history

David at 14:26, 18 July 2014

David: Created page with "== Benchmark using Sockperf == === Check the PCI NIC locality == * Ensure that we are pinning the processes to the cores with the PCI controller [root@hft2 ~]# cat /..."