Difference between revisions of "FhGFS Benchmarking"
Jump to navigation
Jump to search
m (→Test) |
|||
| Line 52: | Line 52: | ||
== Multi Stream Throughput == | == Multi Stream Throughput == | ||
| − | '''Eight clients, filesize of ~1GB per storage server''' | + | <div style="border: 1px #888888 solid; background-color: #e57f52;padding: 3px;">'''Eight clients, filesize of ~1GB per storage server''' </div> |
=== Two Storage Servers === | === Two Storage Servers === | ||
==== Tuning ==== | ==== Tuning ==== | ||
Revision as of 09:58, 7 February 2014
Network Performance
Iperf
Iperf will validate the efficient bandwidth between every machines, the real network connection speed.
root@cloud32:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 172.28.12.32 port 5001 connected with 172.28.12.24 port 40038
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 1.67 GBytes 1.44 Gbits/secroot@cloud24:~/fhgfs-benchmark# iperf -c 172.28.12.32 -i1 -t 10
------------------------------------------------------------
Client connecting to 172.28.12.32, TCP port 5001
TCP window size: 21.6 KByte (default)
------------------------------------------------------------
[ 3] local 172.28.12.24 port 40038 connected with 172.28.12.32 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 1.0- 2.0 sec 172 MBytes 1.44 Gbits/sec
[ 3] 2.0- 3.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 3.0- 4.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 4.0- 5.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 5.0- 6.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 6.0- 7.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 7.0- 8.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 8.0- 9.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 9.0-10.0 sec 171 MBytes 1.44 Gbits/sec
[ 3] 0.0-10.0 sec 1.67 GBytes 1.44 Gbits/secNetcat
Netcat will determine the current bandwidth for the writes.
root@cloud32:~# nc -v -v -l -n 2222 >/dev/null
Listening on [0.0.0.0] (family 0, port 2222)
Connection from [172.28.12.24] port 2222 [tcp/*] accepted (family 2, sport 39647)root@cloud24:~/fhgfs-benchmark# time dd if=/dev/zero | nc -v -v -n 172.28.12.32 2222
Connection to 172.28.12.32 2222 port [tcp/*] succeeded!
^C3884852+0 records in
3884851+0 records out
1989043712 bytes (2.0 GB) copied, 45.9459 s, 43.3 MB/s
real 0m45.953s
user 0m3.719s
sys 1m2.406sMulti Stream Throughput
Eight clients, filesize of ~1GB per storage server
Two Storage Servers
Tuning
Striping across two servers
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=2 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 2
root@cloud28:~# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 2Test
root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 8 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 256m -F -o /mnt/fhgfs/test.iorResult
Summary:
api = POSIX
test filename = /mnt/fhgfs/test.ior
access = file-per-process
ordering in a file = sequential offsets
ordering inter file=constant task offsets = 1
clients = 5 (1 per node)
repetitions = 5
xfersize = 524288 bytes
blocksize = 410 MiB
aggregate filesize = 2.00 GiB
Operation Max (MiB) Min (MiB) Mean (MiB) Std Dev Max (OPs) Min (OPs) Mean (OPs) Std Dev Mean (s)
--------- --------- --------- ---------- ------- --------- --------- ---------- ------- --------
write 137.05 127.16 131.73 3.33 274.10 254.32 263.46 6.67 15.57218 EXCEL
read 334.68 305.41 316.45 10.08 669.35 610.82 632.91 20.16 6.48446 EXCEL
Max Write: 137.05 MiB/sec (143.71 MB/sec)
Max Read: 334.68 MiB/sec (350.93 MB/sec)Four Storage Servers
Tuning
Striping across four storage servers
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=4 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 4
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 4Test
root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 8 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 512m -F -o /mnt/fhgfs/test.iorResult
Summary:
api = POSIX
test filename = /mnt/fhgfs/test.ior
access = file-per-process
ordering in a file = sequential offsets
ordering inter file=constant task offsets = 1
clients = 8 (1 per node)
repetitions = 5
xfersize = 524288 bytes
blocksize = 512 MiB
aggregate filesize = 4 GiB
Operation Max (MiB) Min (MiB) Mean (MiB) Std Dev Max (OPs) Min (OPs) Mean (OPs) Std Dev Mean (s)
--------- --------- --------- ---------- ------- --------- --------- ---------- ------- --------
write 264.27 252.20 258.31 4.50 528.53 504.40 516.63 9.00 15.86156 EXCEL
read 600.25 534.51 564.42 22.51 1200.49 1069.03 1128.85 45.03 7.26843 EXCEL
Max Write: 264.27 MiB/sec (277.10 MB/sec)
Max Read: 600.25 MiB/sec (629.40 MB/sec)Six Storage Servers
Tuning
Striping across six storage servers.
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=6 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 6
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 6Test
root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 8 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 768m -F -o /mnt/fhgfs/test.iorResult
Summary:
api = POSIX
test filename = /mnt/fhgfs/test.ior
access = file-per-process
ordering in a file = sequential offsets
ordering inter file=constant task offsets = 1
clients = 8 (1 per node)
repetitions = 5
xfersize = 524288 bytes
blocksize = 768 MiB
aggregate filesize = 6 GiB
Operation Max (MiB) Min (MiB) Mean (MiB) Std Dev Max (OPs) Min (OPs) Mean (OPs) Std Dev Mean (s)
--------- --------- --------- ---------- ------- --------- --------- ---------- ------- --------
write 360.60 335.26 344.56 8.61 721.20 670.53 689.13 17.23 17.84209 EXCEL
read 629.99 596.70 607.71 11.88 1259.99 1193.40 1215.43 23.77 10.11383 EXCEL
Max Write: 360.60 MiB/sec (378.11 MB/sec)
Max Read: 629.99 MiB/sec (660.60 MB/sec)Eight Storage Servers
Tuning
Striping across eight storage servers.
Number of nodes: 8
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=8 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 8
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 8Test
root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 8 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 1024m -F -o /mnt/fhgfs/test.iorResult
Summary:
api = POSIX
test filename = /mnt/fhgfs/test.ior
access = file-per-process
ordering in a file = sequential offsets
ordering inter file=constant task offsets = 1
clients = 8 (1 per node)
repetitions = 5
xfersize = 524288 bytes
blocksize = 1 GiB
aggregate filesize = 8 GiB
Operation Max (MiB) Min (MiB) Mean (MiB) Std Dev Max (OPs) Min (OPs) Mean (OPs) Std Dev Mean (s)
--------- --------- --------- ---------- ------- --------- --------- ---------- ------- --------
write 371.61 357.65 366.43 4.82 743.21 715.30 732.85 9.63 22.36045 EXCEL
read 648.27 564.96 602.26 27.39 1296.53 1129.92 1204.52 54.78 13.62983 EXCEL
Max Write: 371.61 MiB/sec (389.66 MB/sec)
Max Read: 648.27 MiB/sec (679.76 MB/sec)