FhGFS Benchmarking

From Define Wiki
Revision as of 05:09, 8 August 2013 by Roshan (talk | contribs)
Jump to navigation Jump to search

fhgfs-net

[root@blade1 C]# fhgfs-net

mgmt_nodes
=============
Blade9 [ID: 1]
   Connections: TCP: 1 (101.101.101.9:8008);

meta_nodes
=============
Blade9 [ID: 11201]
   Connections: TCP: 1 (101.101.101.9:8005);

storage_nodes
=============
Blade4 [ID: 3855]
   Connections: TCP: 1 (101.101.101.4:8003);
Blade6 [ID: 17264]
   Connections: TCP: 1 (101.101.101.6:8003);
Blade10 [ID: 18046]
   Connections: TCP: 1 (101.101.101.10:8003);
Blade5 [ID: 32274]
   Connections: TCP: 1 (101.101.101.5:8003);
Blade7 [ID: 41907]
   Connections: TCP: 1 (101.101.101.7:8003);
Blade8 [ID: 56479]
   Connections: TCP: 1 (101.101.101.8:8003);
Blade3 [ID: 60659]
   Connections: TCP: 1 (101.101.101.3:8003);

fhgfs-check-servers

[root@blade1 C]# fhgfs-check-servers
Management
==========
Blade9 [ID: 1]: reachable

Metadata
==========
Blade9 [ID: 11201]: reachable

Storage
==========
Blade4 [ID: 3855]: reachable
Blade6 [ID: 17264]: reachable
Blade10 [ID: 18046]: reachable
Blade5 [ID: 32274]: reachable
Blade7 [ID: 41907]: reachable
Blade8 [ID: 56479]: reachable
Blade3 [ID: 60659]: reachable

fhgfs-df

[root@blade1 C]# fhgfs-df
METADATA SERVERS:
TargetID        Pool       Total        Free      ITotal       IFree
========        ====       =====        ====      ======       =====
   11201    [normal]      49.1GB      46.6GB        3.1M        3.0M

STORAGE TARGETS:
TargetID        Pool       Total        Free      ITotal       IFree
========        ====       =====        ====      ======       =====
    9039       [low]      49.1GB      45.1GB        3.1M        3.1M
   23710       [low]      49.1GB      44.5GB        3.1M        3.0M
   26221       [low]      49.1GB      44.5GB        3.1M        3.1M
   34234       [low]      49.1GB      45.7GB        3.1M        3.1M
   36548       [low]      49.1GB      45.4GB        3.1M        3.0M
   57799       [low]      49.1GB      45.1GB        3.1M        3.1M
   63163       [low]      49.1GB      45.1GB        3.1M        3.1M

IOR

Test 1

[root@blade1 C]# ./IOR
Run began: Tue Aug  6 07:46:08 2013
Command line used: ./IOR
Machine: Linux blade1

Summary:
        api                = POSIX
        test filename      = testFile
        access             = single-shared-file
        ordering in a file = sequential offsets
        ordering inter file= no tasks offsets
        clients            = 1 (1 per node)
        repetitions        = 1
        xfersize           = 262144 bytes
        blocksize          = 1 MiB
        aggregate filesize = 1 MiB

Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)
---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         531.93     531.93      531.93      0.00    2127.74    2127.74     2127.74      0.00   0.00188   EXCEL
read         1326.47    1326.47     1326.47      0.00    5305.89    5305.89     5305.89      0.00   0.00075   EXCEL

Max Write: 531.93 MiB/sec (557.77 MB/sec)
Max Read:  1326.47 MiB/sec (1390.91 MB/sec)

Run finished: Tue Aug  6 07:46:08 2013

Test 2

[root@blade1 C]# ./IOR -vv -k -wWr -C -F -i4 -t 256k -b 10m -s574 -o /mnt/fhgfs/iorfile
Run began: Tue Aug  6 07:35:35 2013
Command line used: ./IOR -vv -k -wWr -C -F -i4 -t 256k -b 10m -s574 -o /mnt/fhgfs/iorfile
Machine: Linux blade1 2.6.32-358.el6.x86_64 #1 SMP Fri Feb 22 00:31:26 UTC 2013 x86_64
Using synchronized MPI timer
Start time skew across all tasks: 0.00 sec
Path: /mnt/fhgfs
FS: 343.6 GiB   Used FS: 6.6%   Inodes: 0.0 Mi   Used Inodes: -nan%
Participating tasks: 1
Using reorderTasks '-C' (expecting block, not cyclic, task assignment)
task 0 on blade1

Summary:
        api                = POSIX
        test filename      = /mnt/fhgfs/iorfile
        access             = file-per-process
        pattern            = strided (574 segments)
        ordering in a file = sequential offsets
        ordering inter file=constant task offsets = 1
        clients            = 1 (1 per node)
        repetitions        = 4
        xfersize           = 262144 bytes
        blocksize          = 10 MiB
        aggregate filesize = 5.61 GiB

Using Time Stamp 1375788940 (0x5200df8c) for Data Signature
Commencing write performance test.
Tue Aug  6 07:35:40 2013

access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s) total(s)  iter
------    ---------  ---------- ---------  --------   --------   --------  --------   ----
write     532.48     10240      256.00     0.000449   10.78      0.000363   10.78      0    XXCEL
Verifying contents of the file(s) just written.
Tue Aug  6 07:35:56 2013

[RANK 000] open for reading file /mnt/fhgfs/iorfile.00000000 XXCEL
Commencing read performance test.
Tue Aug  6 07:36:14 2013

read      475.34     10240      256.00     0.001303   12.07      0.000385   12.08      0    XXCEL
Using Time Stamp 1375788986 (0x5200dfba) for Data Signature
Commencing write performance test.
Tue Aug  6 07:36:26 2013

write     270.84     10240      256.00     0.238056   20.96      0.000310   21.19      1    XXCEL
Verifying contents of the file(s) just written.
Tue Aug  6 07:36:47 2013

[RANK 000] open for reading file /mnt/fhgfs/iorfile.00000000 XXCEL
Commencing read performance test.
Tue Aug  6 07:37:04 2013

read      496.80     10240      256.00     0.001265   11.55      0.000352   11.55      1    XXCEL
Using Time Stamp 1375789036 (0x5200dfec) for Data Signature
Commencing write performance test.
Tue Aug  6 07:37:16 2013

write     217.72     10240      256.00     0.270669   26.09      0.000319   26.36      2    XXCEL
Verifying contents of the file(s) just written.
Tue Aug  6 07:37:42 2013

[RANK 000] open for reading file /mnt/fhgfs/iorfile.00000000 XXCEL
Commencing read performance test.
Tue Aug  6 07:38:00 2013

read      497.94     10240      256.00     0.001282   11.53      0.000351   11.53      2    XXCEL
Using Time Stamp 1375789091 (0x5200e023) for Data Signature
Commencing write performance test.
Tue Aug  6 07:38:11 2013

write     219.26     10240      256.00     0.235869   25.94      0.000313   26.18      3    XXCEL
Verifying contents of the file(s) just written.
Tue Aug  6 07:38:37 2013

[RANK 000] open for reading file /mnt/fhgfs/iorfile.00000000 XXCEL
Commencing read performance test.
Tue Aug  6 07:38:55 2013

read      468.33     10240      256.00     0.001103   12.25      0.000376   12.26      3    XXCEL
Operation  Max (MiB)  Min (MiB)  Mean (MiB)   Std Dev  Max (OPs)  Min (OPs)  Mean (OPs)   Std Dev  Mean (s)  Op grep #Tasks tPN reps  fPP reord reordoff reordrand seed segcnt blksiz xsize aggsize

---------  ---------  ---------  ----------   -------  ---------  ---------  ----------   -------  --------
write         532.48     217.72      310.07    130.17       3.71       1.52        2.16      0.91  21.12902   1 1 4 1 1 1 0 0 574 10485760 262144 6018826240 -1 POSIX EXCEL
read          497.94     468.33      484.60     13.01       3.47       3.26        3.38      0.09  11.85335   1 1 4 1 1 1 0 0 574 10485760 262144 6018826240 -1 POSIX EXCEL

Max Write: 532.48 MiB/sec (558.35 MB/sec)
Max Read:  497.94 MiB/sec (522.12 MB/sec)

Run finished: Tue Aug  6 07:39:07 2013

MDTEST

Test 1

-- started at 08/06/2013 07:49:58 --

mdtest-1.9.1 was launched with 1 total task(s) on 1 node(s)
Command line used: ./mdtest -n 1024 -d /mnt/fhgfs/MDTEST -F -C -T -r -N 1
Path: /mnt/fhgfs
FS: 343.6 GiB   Used FS: 8.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

1 tasks, 1024 files

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :       2071.708       2071.708       2071.708          0.000
   File stat         :       6424.898       6424.898       6424.898          0.000
   File read         :          0.000          0.000          0.000          0.000
   File removal      :       1879.944       1879.944       1879.944          0.000
   Tree creation     :       1157.688       1157.688       1157.688          0.000
   Tree removal      :        790.483        790.483        790.483          0.000

-- finished at 08/06/2013 07:49:59 --

Test 2

-- started at 08/06/2013 07:51:56 --

mdtest-1.9.1 was launched with 1 total task(s) on 1 node(s)
Command line used: ./mdtest -n 1024 -d /mnt/fhgfs/MDTEST -S -C -T -r -N 1
Path: /mnt/fhgfs
FS: 343.6 GiB   Used FS: 8.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

1 tasks, 1 file

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :       2737.946       2737.946       2737.946          0.000
   File stat         :        199.851        199.851        199.851          0.000
   File read         :          0.000          0.000          0.000          0.000
   File removal      :       2503.440       2503.440       2503.440          0.000
   Tree creation     :       1277.194       1277.194       1277.194          0.000
   Tree removal      :       1117.289       1117.289       1117.289          0.000

-- finished at 08/06/2013 07:52:02 --

Test 3

-- started at 08/06/2013 07:53:42 --

mdtest-1.9.1 was launched with 2 total task(s) on 1 node(s)
Command line used: ./mdtest -n 1024 -d /mnt/fhgfs/MDTEST -S -C -T -r -n 1
Path: /mnt/fhgfs
FS: 343.6 GiB   Used FS: 8.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

2 tasks, 1 file

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :          0.400          0.400          0.400          0.000
   File stat         :       1558.931       1558.931       1558.931          0.000
   File read         :          0.000          0.000          0.000          0.000
   File removal      :       3718.355       3718.355       3718.355          0.000
   Tree creation     :       1237.623       1237.623       1237.623          0.000
   Tree removal      :        896.027        896.027        896.027          0.000

-- finished at 08/06/2013 07:53:47 --

Test 4

-- started at 08/06/2013 07:55:05 --

mdtest-1.9.1 was launched with 10 total task(s) on 1 node(s)
Command line used: ./mdtest -n 1024 -d /mnt/fhgfs/MDTEST -S -C -T -r -n 1
Path: /mnt/fhgfs
FS: 343.6 GiB   Used FS: 8.3%   Inodes: 0.0 Mi   Used Inodes: -nan%

10 tasks, 1 file

SUMMARY: (of 1 iterations)
   Operation                      Max            Min           Mean        Std Dev
   ---------                      ---            ---           ----        -------
   File creation     :       1563.231       1563.231       1563.231          0.000
   File stat         :      10988.483      10988.483      10988.483          0.000
   File read         :          0.000          0.000          0.000          0.000
   File removal      :      23750.306      23750.306      23750.306          0.000
   Tree creation     :       1524.647       1524.647       1524.647          0.000
   Tree removal      :       1058.366       1058.366       1058.366          0.000

-- finished at 08/06/2013 07:55:05 --
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=FhGFS_Benchmarking&oldid=2969"