Difference between revisions of "FhGFS Benchmarking"

From Define Wiki
Jump to navigation Jump to search
Line 184: Line 184:
 
         ordering in a file = sequential offsets
 
         ordering in a file = sequential offsets
 
         ordering inter file=constant task offsets = 1
 
         ordering inter file=constant task offsets = 1
−
         clients            = 5 (1 per node)
+
         clients            = 8 (1 per node)
 
         repetitions        = 5
 
         repetitions        = 5
 
         xfersize          = 524288 bytes
 
         xfersize          = 524288 bytes
−
         blocksize          = 1.60 GiB
+
         blocksize          = 1 GiB
−
         aggregate filesize = 8.00 GiB
+
         aggregate filesize = 8 GiB
  
 
Operation  Max (MiB)  Min (MiB)  Mean (MiB)  Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)  Std Dev  Mean (s)
 
Operation  Max (MiB)  Min (MiB)  Mean (MiB)  Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)  Std Dev  Mean (s)
 
---------  ---------  ---------  ----------  -------  ---------  ---------  ----------  -------  --------
 
---------  ---------  ---------  ----------  -------  ---------  ---------  ----------  -------  --------
−
write        268.02     256.55     261.34     3.98     536.04     513.10     522.68     7.95 31.36454   EXCEL
+
write        371.61     357.65     366.43     4.82     743.21     715.30     732.85     9.63 22.36045   EXCEL
−
read          453.39     427.66     443.72      8.62    906.79     855.32      887.45     17.24 18.47582   EXCEL
+
read          648.27     564.96     602.26    27.39    1296.53    1129.92     1204.52     54.78 13.62983   EXCEL
  
−
Max Write: 268.02 MiB/sec (281.04 MB/sec)
+
Max Write: 371.61 MiB/sec (389.66 MB/sec)
−
Max Read:  453.39 MiB/sec (475.42 MB/sec)
+
Max Read:  648.27 MiB/sec (679.76 MB/sec)
 
</syntaxhighlight>
 
</syntaxhighlight>

Revision as of 09:17, 7 February 2014

Network Performance

Iperf

Iperf will validate the efficient bandwidth between every machines, the real network connection speed. 

Server
Client

Multi Stream Throughput

5 clients, filesize of ~1GB per storage server

Two Storage Servers

Tuning

Striping across two servers 

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=2 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 2

root@cloud28:~# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 2

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 5 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 410m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 5 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 410 MiB
        aggregate filesize = 2.00 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         137.05     127.16      131.73      3.33     274.10     254.32      263.46      6.67  15.57218   EXCEL
read          334.68     305.41      316.45     10.08     669.35     610.82      632.91     20.16   6.48446   EXCEL

Max Write: 137.05 MiB/sec (143.71 MB/sec)
Max Read:  334.68 MiB/sec (350.93 MB/sec)

Four Storage Servers

Tuning

Striping across four storage servers 

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=4 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 4

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 4

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 5 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 820m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 5 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 820 MiB
        aggregate filesize = 4.00 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         236.70     227.91      232.03      2.97     473.41     455.83      464.05      5.93  17.67324   EXCEL
read          417.67     355.39      392.24     22.96     835.34     710.79      784.49     45.91  10.48953   EXCEL

Max Write: 236.70 MiB/sec (248.20 MB/sec)
Max Read:  417.67 MiB/sec (437.96 MB/sec)

Six Storage Servers

Tuning

Striping across six storage servers. 

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=6 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 6

root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 6

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 5 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 1229m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 5 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 1.20 GiB
        aggregate filesize = 6.00 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         253.46     246.93      250.55      2.26     506.93     493.86      501.11      4.52  24.52773   EXCEL
read          403.10     380.50      392.76      7.41     806.20     761.00      785.52     14.83  15.65139   EXCEL

Max Write: 253.46 MiB/sec (265.77 MB/sec)
Max Read:  403.10 MiB/sec (422.68 MB/sec)

Eight Storage Servers

Tuning

Striping across eight storage servers. 

Number of nodes: 8
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --setpattern --numtargets=8 --chunksize=512k /mnt/fhgfs
Mount: '/mnt/fhgfs'; Path: ''
Chunksize: 524288
Number of storage targets: 8
root@cloud24:~/fhgfs-benchmark# fhgfs-ctl --getentryinfo /mnt/fhgfs
Path:
Mount: /mnt/fhgfs
EntryID: root
Metadata node: cloud32 [ID: 13708]
Stripe pattern details:
+ Type: RAID0
+ Chunksize: 512K
+ Number of storage targets: desired: 8

Test

root@cloud24:~/fhgfs-benchmark# mpirun -hostfile hostfile -np 8 /root/ior/IOR/src/C/IOR -wr -C -i5 -t512k -b 1024m -F -o /mnt/fhgfs/test.ior

Result

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/test.ior
        access             = file-per-process
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 8 (1 per node)
        repetitions        = 5
        xfersize           = 524288 bytes
        blocksize          = 1 GiB
        aggregate filesize = 8 GiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         371.61     357.65      366.43      4.82     743.21     715.30      732.85      9.63  22.36045   EXCEL
read          648.27     564.96      602.26     27.39    1296.53    1129.92     1204.52     54.78  13.62983   EXCEL

Max Write: 371.61 MiB/sec (389.66 MB/sec)
Max Read:  648.27 MiB/sec (679.76 MB/sec)
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=FhGFS_Benchmarking&oldid=3947"