Difference between revisions of "Benchmarking: IOZone (CERN)"

From Define Wiki
Jump to navigation Jump to search
 
(33 intermediate revisions by 2 users not shown)
Line 77: Line 77:
  
 
== Benchmarking a single disk ==
 
== Benchmarking a single disk ==
−
This test is to measure the performance of the first drive in the disk array. Cern ask for the first and last disks in the storage array to be tested and for each test to the run 4 times and an average be taken.
+
Cern ask for the first and last disks in the storage array to be tested and for each test to be run 4 times and an average be taken.
 
<br><br>
 
<br><br>
 
This benchmark can be scripted as follows:
 
This benchmark can be scripted as follows:
Line 91: Line 91:
 
And then averages calculated:
 
And then averages calculated:
 
<syntaxhighlight>
 
<syntaxhighlight>
 +
echo " "
 +
echo "*** Results in KB/s ***"
 +
echo " "
 
echo "1st Disk"
 
echo "1st Disk"
 
echo "=========="
 
echo "=========="
Line 98: Line 101:
 
grep 'Random read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1  
−
 
+
echo " "
 
echo "16th Disk"
 
echo "16th Disk"
 
echo "=========="
 
echo "=========="
Line 107: Line 110:
 
grep 'Random write'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1
 
grep 'Random write'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
== Benchmarking all disks ==
 +
Cern also ask for all disks in the storage array to be tested together and, again, for each test to be run 4 times and an average be taken.
 +
<br><br>
 +
This can be scripted as follows (in this case, with 24 drives being tested):
 +
<syntaxhighlight>
 +
for i in `seq 1 4`
 +
do
 +
/opt/iozone/bin/iozone -R -I -l 24 -u 24 -r 1m -s 4g -F /srv/castor/{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24}/iozone.dat | tee -a iozone-1m-4g-24fs-loop-"$i".txt
 +
done
 +
</syntaxhighlight>
 +
For a different total number of drives, adjust the '''<code>-l 24</code>''' and '''<code>-u 24</code>''' values accordingly.
 +
<br><br>
 +
And then averages calculated:
 +
<syntaxhighlight>
 +
echo " "
 +
echo "*** Results in KB/s ***"
 +
echo " "
 +
grep '  Read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1
 +
grep 'Initial write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1
 +
grep 'Random read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1
 +
grep 'Random write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1
 +
</syntaxhighlight>
 +
 +
== Single Disk Results ==
 +
{| class="wikitable sortable" cellpadding=2 style="border:1px; width:100%; font-size:50%; text-align: center;"
 +
|-
 +
! colspan="5" | System Config || colspan="4" | 1st Disk (KB/s) || colspan="4" | Last Disk (KB/s)
 +
|-
 +
! Disk Enclosure || Controller || Disk Model || Disk Type || Spindle Speed || Read || Initial Write || Rand. Read || Rand. Write || Read || Initial Write || Rand. Read || Rand. Write || Tender
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || ST6000NM0024 || 6Gb/s SATA || 7,200 || 220139 || 212904 || 100416 || 147607 || 220766 || 220254 || 99381 || 147636 || IT-4032
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || HUS726060ALA640 || 6Gb/s SATA || 7,200 || 184203 || 184153 || 87583 || 90935 || 183946 || 183679 || 87005 || 89375 || IT-4032
 +
|}
 +
 +
== Multiple Disk Results ==
 +
{| class="wikitable sortable" cellpadding=2 style="border:1px; width:100%; font-size:50%; text-align: center;"
 +
|-
 +
! colspan="6" | System Config || colspan="4" | All Disks Combined (KB/s)
 +
|-
 +
! Disk Enclosure || Controller || Disk Model || Disk Type || Spindle Speed || Qty || Read || Initial Write || Rand. Read || Rand. Write || Tender
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || ST6000NM0024 || 6Gb/s SATA || 7,200 || 16 || 1,872,830 || 1,699,285 || 1,321,172 || 1,743,663 || IT-4032
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || HUS726060ALA640 (FW 1EN) || 6Gb/s SATA || 7,200 || 16 || 1,894,552 || 1,598,613 || 1,301,298 || 1,370,605 || IT-4032
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || HUS726060ALA640 (FW 1EQ) || 6Gb/s SATA || 7,200 || 16 || 2,082,375 || 1,755,122 || 1,332,840 || 1,393,900|| IT-4032
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || WD4000F9YZ (FW AA2) || 6Gb/s SATA || 7,200 || 24 || 1,918,817 || 1,659,493 || 1,282,772 || 1,390,060|| IT-4032
 +
|-
 +
| Supermicro #### || LSI 9207-8e (2 Ports Connected) || WD6005FRPZ (FW ###) || 6Gb/s SATA || 7,200 || 24 || 4,025,574 || 3,761,571 || N/A || N/A || IT-4138
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || HUS726060ALA640) (FW 1EQ) || 6Gb/s SATA || 7,200 || 24 || 2,008,925 || 1,704,049 || N/A  || N/A|| IT-4138
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || HUS726060ALE610 (FW 517)|| 6Gb/s SATA || 7,200 || 24 || 2,068,481 || 1,755,754 || N/A  || N/A|| IT-4138
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || ### || 6Gb/s SATA || 7,200 || 24 || 0 || 0 || N/A  || N/A|| IT-4138
 +
|-
 +
| Supermicro 2028R-NR48N || N/A || Intel P3500 || NVMe || N/A || 48 || 25,622,050 || 27,792,330 || 25,639,068 || 27,782,964 || Just for fun!
 +
|}

Latest revision as of 13:12, 23 September 2016

Prerequisites

XFS & lsscsi need installing: 

yum install xfsprogs.x86_64 lsscsi

As does IOZONE: 

yum install iozone

Preparing the data filesystems

All available block devices in the attached disk enclosure will be used for this test. None of the drives in the front-end server will be used.

Get the list of available block devices

Use lsscsi to identify the drives to be tested (in this case, /dev/sdc to /dev/sdr): 

[root@nodea ~]# lsscsi
[0:0:0:0]    enclosu Promise  J630s            060=  -       
[0:0:1:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdc 
[0:0:2:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdd 
[0:0:3:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sde 
[0:0:4:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdf 
[0:0:5:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdg 
[0:0:6:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdh 
[0:0:7:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdi 
[0:0:8:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdj 
[0:0:9:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdk 
[0:0:10:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdl 
[0:0:11:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdm 
[0:0:12:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdn 
[0:0:13:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdo 
[0:0:14:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdp 
[0:0:15:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdq 
[0:0:16:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdr 
[1:0:0:0]    disk    ATA      Hitachi HUA72302 MK7O  /dev/sda 
[2:0:0:0]    disk    ATA      Hitachi HUA72302 MK7O  /dev/sdb

Create an XFS filesystem on each drive

The basic command for this is: 

mkfs.xfs -l version=2 -i size=1024 -n size=16384 -d su=64k,sw=1 <block device>

So can be scripted as follows: 

for i in {c..r};
do
	mkfs.xfs -l version=2 -i size=1024 -n size=16384 -d su=64k,sw=1 /dev/sd${i} &
done

Create mount points

Create mount points in /srv/castor/XY where XY ranges from 1 to the number of available block devices. For example, it will be 16 on a 16-bay disk enclosure.
This can be scripted as follows: 

mkdir /srv/castor
for i in {1..16};
do
	mkdir /srv/castor/${i}
done

Mount filesystems to mount points

Mount these filesystems using the following options: logbufs=8,logbsize=256k,noatime,swallow,inode64
This can be scripted to add mount point to /etc/fstab as follows: 

n=1

for i in {c..r}
do
	echo "/dev/sd${i}	/srv/castor/${n}	xfs logbufs=8,logbsize=256k,noatime,swalloc,inode64" >> /etc/fstab
	((n++))
done

Benchmarking a single disk

Cern ask for the first and last disks in the storage array to be tested and for each test to be run 4 times and an average be taken.

This benchmark can be scripted as follows: 

for j in `seq 1 4`;
do
	for i in {1,16}; # << Change 16 to the total number of drives being tested
	do
		/opt/iozone/bin/iozone -i 0 -i 1 -i 2 -R -I -l 1 -u 1 -r 1m -s 4g -F /srv/castor/"$i"/iozone.dat | tee -a iozone-1m-4g-single-disk-"$i"-loop-"$j".txt
	done
done

And then averages calculated: 

echo " "
echo "*** Results in KB/s ***"
echo " "
echo "1st Disk"
echo "=========="
echo " "
grep '  Read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1 
grep 'Initial write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1 
grep 'Random read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1 
grep 'Random write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1 
echo " "
echo "16th Disk"
echo "=========="
echo " "
grep '  Read'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1
grep 'Initial write'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1
grep 'Random read'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1
grep 'Random write'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1

Benchmarking all disks

Cern also ask for all disks in the storage array to be tested together and, again, for each test to be run 4 times and an average be taken.

This can be scripted as follows (in this case, with 24 drives being tested): 

for i in `seq 1 4`
do
	/opt/iozone/bin/iozone -R -I -l 24 -u 24 -r 1m -s 4g -F /srv/castor/{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24}/iozone.dat | tee -a iozone-1m-4g-24fs-loop-"$i".txt
done

For a different total number of drives, adjust the -l 24 and -u 24 values accordingly.

And then averages calculated: 

echo " "
echo "*** Results in KB/s ***"
echo " "
grep '  Read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1 
grep 'Initial write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1 
grep 'Random read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1 
grep 'Random write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1

Single Disk Results

System Config 1st Disk (KB/s) Last Disk (KB/s)
Disk Enclosure Controller Disk Model Disk Type Spindle Speed Read Initial Write Rand. Read Rand. Write Read Initial Write Rand. Read Rand. Write Tender
Promise VTRAK J630S LSI 9207-8e ST6000NM0024 6Gb/s SATA 7,200 220139 212904 100416 147607 220766 220254 99381 147636 IT-4032
Promise VTRAK J630S LSI 9207-8e HUS726060ALA640 6Gb/s SATA 7,200 184203 184153 87583 90935 183946 183679 87005 89375 IT-4032

Multiple Disk Results

System Config All Disks Combined (KB/s)
Disk Enclosure Controller Disk Model Disk Type Spindle Speed Qty Read Initial Write Rand. Read Rand. Write Tender
Promise VTRAK J630S LSI 9207-8e ST6000NM0024 6Gb/s SATA 7,200 16 1,872,830 1,699,285 1,321,172 1,743,663 IT-4032
Promise VTRAK J630S LSI 9207-8e HUS726060ALA640 (FW 1EN) 6Gb/s SATA 7,200 16 1,894,552 1,598,613 1,301,298 1,370,605 IT-4032
Promise VTRAK J630S LSI 9207-8e HUS726060ALA640 (FW 1EQ) 6Gb/s SATA 7,200 16 2,082,375 1,755,122 1,332,840 1,393,900 IT-4032
Promise VTRAK J830S LSI 9207-8e WD4000F9YZ (FW AA2) 6Gb/s SATA 7,200 24 1,918,817 1,659,493 1,282,772 1,390,060 IT-4032
Supermicro #### LSI 9207-8e (2 Ports Connected) WD6005FRPZ (FW ###) 6Gb/s SATA 7,200 24 4,025,574 3,761,571 N/A N/A IT-4138
Promise VTRAK J830S LSI 9207-8e HUS726060ALA640) (FW 1EQ) 6Gb/s SATA 7,200 24 2,008,925 1,704,049 N/A N/A IT-4138
Promise VTRAK J830S LSI 9207-8e HUS726060ALE610 (FW 517) 6Gb/s SATA 7,200 24 2,068,481 1,755,754 N/A N/A IT-4138
Promise VTRAK J830S LSI 9207-8e ### 6Gb/s SATA 7,200 24 0 0 N/A N/A IT-4138
Supermicro 2028R-NR48N N/A Intel P3500 NVMe N/A 48 25,622,050 27,792,330 25,639,068 27,782,964 Just for fun!
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=Benchmarking:_IOZone_(CERN)&oldid=16881"