Difference between revisions of "FhGFS Benchmarking"

From Define Wiki
Jump to navigation Jump to search
Line 3: Line 3:
 
Iperf will validate the efficient bandwidth between every machines, the real network connection speed.
 
Iperf will validate the efficient bandwidth between every machines, the real network connection speed.
 
<syntaxhighlight>
 
<syntaxhighlight>
−
Server
+
root@cloud32:~# iperf -s
 +
------------------------------------------------------------
 +
Server listening on TCP port 5001
 +
TCP window size: 85.3 KByte (default)
 +
------------------------------------------------------------
 +
[  4] local 172.28.12.32 port 5001 connected with 172.28.12.24 port 40038
 +
[ ID] Interval      Transfer    Bandwidth
 +
[  4]  0.0-10.0 sec  1.67 GBytes  1.44 Gbits/sec
 
</syntaxhighlight>
 
</syntaxhighlight>
 
<syntaxhighlight>
 
<syntaxhighlight>
−
Client
+
root@cloud24:~/fhgfs-benchmark# iperf -c 172.28.12.32 -i1 -t 10
 +
------------------------------------------------------------
 +
Client connecting to 172.28.12.32, TCP port 5001
 +
TCP window size: 21.6 KByte (default)
 +
------------------------------------------------------------
 +
[  3] local 172.28.12.24 port 40038 connected with 172.28.12.32 port 5001
 +
[ ID] Interval      Transfer    Bandwidth
 +
[  3]  0.0- 1.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  1.0- 2.0 sec  172 MBytes  1.44 Gbits/sec
 +
[  3]  2.0- 3.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  3.0- 4.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  4.0- 5.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  5.0- 6.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  6.0- 7.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  7.0- 8.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  8.0- 9.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  9.0-10.0 sec  171 MBytes  1.44 Gbits/sec
 +
[  3]  0.0-10.0 sec  1.67 GBytes  1.44 Gbits/sec
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 
== Multi Stream Throughput ==
 
== Multi Stream Throughput ==
 
'''5 clients, filesize of ~1GB per storage server'''  
 
'''5 clients, filesize of ~1GB per storage server'''  

Revision as of 09:20, 7 February 2014

Network Performance

Iperf

Iperf will validate the efficient bandwidth between every machines, the real network connection speed. 

root@cloud32:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 172.28.12.32 port 5001 connected with 172.28.12.24 port 40038
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  1.67 GBytes  1.44 Gbits/sec
root@cloud24:~/fhgfs-benchmark# iperf -c 172.28.12.32 -i1 -t 10
------------------------------------------------------------
Client connecting to 172.28.12.32, TCP port 5001
TCP window size: 21.6 KByte (default)
------------------------------------------------------------
[  3] local 172.28.12.24 port 40038 connected with 172.28.12.32 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  1.0- 2.0 sec   172 MBytes  1.44 Gbits/sec
[  3]  2.0- 3.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  3.0- 4.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  4.0- 5.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  5.0- 6.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  6.0- 7.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  7.0- 8.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  8.0- 9.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  9.0-10.0 sec   171 MBytes  1.44 Gbits/sec
[  3]  0.0-10.0 sec  1.67 GBytes  1.44 Gbits/sec

Multi Stream Throughput

5 clients, filesize of ~1GB per storage server

Two Storage Servers

Tuning

Striping across two servers 

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=2 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 2

root@cloud28:~# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 2

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 5 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 410m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 5 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 410 MiB
        aggregate filesize = 2.00 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         137.05     127.16      131.73      3.33     274.10     254.32      263.46      6.67  15.57218   EXCEL
read          334.68     305.41      316.45     10.08     669.35     610.82      632.91     20.16   6.48446   EXCEL

Max Write: 137.05 MiB/sec (143.71 MB/sec)
Max Read:  334.68 MiB/sec (350.93 MB/sec)

Four Storage Servers

Tuning

Striping across four storage servers 

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=4 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 4

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 4

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 5 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 820m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 5 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 820 MiB
        aggregate filesize = 4.00 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         236.70     227.91      232.03      2.97     473.41     455.83      464.05      5.93  17.67324   EXCEL
read          417.67     355.39      392.24     22.96     835.34     710.79      784.49     45.91  10.48953   EXCEL

Max Write: 236.70 MiB/sec (248.20 MB/sec)
Max Read:  417.67 MiB/sec (437.96 MB/sec)

Six Storage Servers

Tuning

Striping across six storage servers. 

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=6 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 6

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 6

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 5 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 1229m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 5 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 1.20 GiB
        aggregate filesize = 6.00 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         253.46     246.93      250.55      2.26     506.93     493.86      501.11      4.52  24.52773   EXCEL
read          403.10     380.50      392.76      7.41     806.20     761.00      785.52     14.83  15.65139   EXCEL

Max Write: 253.46 MiB/sec (265.77 MB/sec)
Max Read:  403.10 MiB/sec (422.68 MB/sec)

Eight Storage Servers

Tuning

Striping across eight storage servers. 

Number of nodes: 8
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=8 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 8
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 8

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 8 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 1024m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 8 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 1 GiB
        aggregate filesize = 8 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         371.61     357.65      366.43      4.82     743.21     715.30      732.85      9.63  22.36045   EXCEL
read          648.27     564.96      602.26     27.39    1296.53    1129.92     1204.52     54.78  13.62983   EXCEL

Max Write: 371.61 MiB/sec (389.66 MB/sec)
Max Read:  648.27 MiB/sec (679.76 MB/sec)
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=FhGFS_Benchmarking&oldid=3948"