Benchmarking: STFC Storage (GridPP) with Iozone

• Notes below are based on the 2014 test suits (these tests have been fairly consistent over the years)

Setup the RAID Array

• 22 Drives RAID6 array with WD SE WD4000F9YS Drives
• E5-2620v2 CPUs
• 64GB RAM
• LSI 9261 Controller with the following settings:
  • RAID Level: 6
  • Stripe Size: 256k
  • Disk Cache Policy: Enabled
  • Read Policy: Always Read Ahead
  • Write Policy: Always Write Back
  • IO Policy: Direct IO

Partition and Create FS

[root@localhost scripts]# cat intelligent-partitioner-dp.sh 
#!/bin/bash

# Parted stuff
parted -a optimal /dev/sda unit % mkpart primary 0% 33%    ; sleep 1
parted -a optimal /dev/sda unit % mkpart primary 33% 66%   ; sleep 1
parted -a optimal /dev/sda unit % mkpart primary 66% 100%  ; sleep 1

# sleep for 5
sleep 5

partprobe /dev/sda

sleep 5
# Create the XFS Part
mkfs.xfs -f -l version=2 -i size=1024 -n size=65536 -d su=256k,sw=22 -L castor1 /dev/sda1
sleep 2

mkfs.xfs -f -l version=2 -i size=1024 -n size=65536 -d su=256k,sw=22 -L castor2 /dev/sda2
sleep 2

mkfs.xfs -f -l version=2 -i size=1024 -n size=65536 -d su=256k,sw=22 -L castor3 /dev/sda3
sleep 2

Mount the Filesystems - add the follwing to fstab

# fstab
/dev/disk/by-label/castor1     /exportstage/castor1  xfs  logbufs=8,logbsize=256k,noatime,swalloc,inode64  0 2
/dev/disk/by-label/castor2     /exportstage/castor2  xfs  logbufs=8,logbsize=256k,noatime,swalloc,inode64  0 2
/dev/disk/by-label/castor3     /exportstage/castor3  xfs  logbufs=8,logbsize=256k,noatime,swalloc,inode64  0 2

Tuning on the System

• Set the following
  • Readahead
  • Request Queue
  • Scheduler

[root@localhost dp]# cat check_settings.sh 
#!/bin/bash

echo -en "[readahead]: "
blockdev --getra /dev/sda
echo -en "[nr_requests]: "
cat /sys/block/sda/queue/nr_requests
echo "[scheduler]: "
cat /sys/block/sda/queue/scheduler
[root@localhost dp]# ./check_settings.sh 
[readahead]: 16384
[nr_requests]: 512
[scheduler]: 
noop anticipatory [deadline] cfq