Memblaze Cards

Driver Installation

All drivers were provided with teh cards and a copy can be found on storage1 in the hpc folder.

Cd into the driver folder and simply run these two commands:

make 
make install

Check cards are detected

in the tools folder run the memmonitor to see the status of the cards:

Memblaze Monitor  - Memblaze PCIE Accelerator Monitor

(c) 2014 Memblaze Technology Co., Ltd (All Rights Reserved)
Basic Information
------------------------------------------------------------------------------------------------------------------------------------------------------
Manufacture Name:               Memblaze Technology Co.,Ltd
Product Name:                   PBlaze3
Flash Cell:                     MLC
Model:                          PB3LM1.20T01
Form Factor:                    Half length, half height
Serial Number:                  MH13240245
Raw Capacity:                   1.68TB
Available Capacity:             1.20TB
Maximum Capacity:               1.20TB
Max Write Capacity:             8.38PB
Already Read Capacity:          336.11TB
Already Write Capacity:         262.27TB
Write Amplification:            1.74

Device Information
------------------------------------------------------------------------------------------------------------------------------------------------------
Name:                           /dev/memdiska
Driver:                         01.05.4083RTM (Compiled on Feb 10 2015 11:30:45), 355.01kB
Firmware:                       01.05.4073RTM (Compiled on Jul 24 2014 16:01:15), 12.58MB
Initialization Counts:          3
Starup Counts:                  34
PCIe Link:                      x8(Gen2)
Status:                         Online
Read only:                      No
Alert Code:                     0
Device Remain Life:             |>>------------------|  (94.57%)

Performance Information
------------------------------------------------------------------------------------------------------------------------------------------------------
                                <6.96us         6.96us~27.8us   27.8us~111.3us  111.3us~445.2us 445.2us~1.78ms  1.78ms~7.12ms   7.12ms~28.5ms   >28.5m

Verify the Performance

• Use FIO for IOPS and Bandwidth tests
• Read IOPS: ~750k IOPS
• BW:

• Read IOPS Test

[root@haswell9 ~]# fio --name=readiops --filename=/dev/memdiska --direct=1 --rw=randread --bs=4096 --numjobs=8 --iodepth=32 --direct=1 --iodepth_batch=16 --iodepth_batch_complete=16 --runtime=300 --ramp_time=5 --norandommap --time_based --ioengine=libaio --group_reporting
readiops: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
...
readiops: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=32
fio-2.0.13
Starting 8 processes
Jobs: 8 (f=8): [rrrrrrrr] [100.0% done] [2755M/0K/0K /s] [705K/0 /0  iops] [eta 00m:00s]
readiops: (groupid=0, jobs=8): err= 0: pid=13212: Tue Feb 17 12:56:24 2015
  read : io=870175MB, bw=2900.6MB/s, iops=742545 , runt=300001msec
    slat (usec): min=18 , max=281509 , avg=99.04, stdev=227.66
    clat (usec): min=1 , max=281668 , avg=242.03, stdev=365.13
     lat (usec): min=38 , max=365377 , avg=341.34, stdev=451.37
    clat percentiles (usec):
     |  1.00th=[    1],  5.00th=[   45], 10.00th=[   79], 20.00th=[  126],
     | 30.00th=[  163], 40.00th=[  191], 50.00th=[  213], 60.00th=[  235],
     | 70.00th=[  262], 80.00th=[  298], 90.00th=[  390], 95.00th=[  524],
     | 99.00th=[  996], 99.50th=[ 1368], 99.90th=[ 2832], 99.95th=[ 3888],
     | 99.99th=[ 8512]
    bw (KB/s)  : min=   12, max=486656, per=12.48%, avg=370822.29, stdev=33550.30
    lat (usec) : 2=1.13%, 4=1.74%, 10=0.15%, 20=0.32%, 50=2.13%
    lat (usec) : 100=8.67%, 250=51.90%, 500=28.43%, 750=3.66%, 1000=0.88%
    lat (msec) : 2=0.78%, 4=0.17%, 10=0.04%, 20=0.01%, 50=0.01%
    lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
  cpu          : usr=3.54%, sys=64.44%, ctx=66465371, majf=0, minf=470
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.1%, 32=101.6%, >=64=0.0%
     submit    : 0=0.0%, 4=0.0%, 8=0.0%, 16=100.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=0.0%, 8=0.0%, 16=100.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued    : total=r=222764496/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
   READ: io=870175MB, aggrb=2900.6MB/s, minb=2900.6MB/s, maxb=2900.6MB/s, mint=300001msec, maxt=300001msec

Disk stats (read/write):
  memdiska: ios=226196097/0, merge=0/0, ticks=41615182/0, in_queue=45701588, util=100.00%