Difference between revisions of "Benchmarking: IOZone (CERN)"

From Define Wiki
Jump to navigation Jump to search
 
(31 intermediate revisions by 2 users not shown)
Line 91: Line 91:
 
And then averages calculated:
 
And then averages calculated:
 
<syntaxhighlight>
 
<syntaxhighlight>
 +
echo " "
 +
echo "*** Results in KB/s ***"
 +
echo " "
 
echo "1st Disk"
 
echo "1st Disk"
 
echo "=========="
 
echo "=========="
Line 98: Line 101:
 
grep 'Random read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1  
−
 
+
echo " "
 
echo "16th Disk"
 
echo "16th Disk"
 
echo "=========="
 
echo "=========="
Line 118: Line 121:
 
done
 
done
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
For a different total number of drives, adjust the '''<code>-l 24</code>''' and '''<code>-u 24</code>''' values accordingly.
 +
<br><br>
 
And then averages calculated:
 
And then averages calculated:
 
<syntaxhighlight>
 
<syntaxhighlight>
 +
echo " "
 +
echo "*** Results in KB/s ***"
 +
echo " "
 
grep '  Read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1  
 
grep '  Read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1  
 
grep 'Initial write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1  
 
grep 'Initial write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1  
Line 125: Line 133:
 
grep 'Random write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1  
 
grep 'Random write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1  
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
== Single Disk Results ==
 +
{| class="wikitable sortable" cellpadding=2 style="border:1px; width:100%; font-size:50%; text-align: center;"
 +
|-
 +
! colspan="5" | System Config || colspan="4" | 1st Disk (KB/s) || colspan="4" | Last Disk (KB/s)
 +
|-
 +
! Disk Enclosure || Controller || Disk Model || Disk Type || Spindle Speed || Read || Initial Write || Rand. Read || Rand. Write || Read || Initial Write || Rand. Read || Rand. Write || Tender
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || ST6000NM0024 || 6Gb/s SATA || 7,200 || 220139 || 212904 || 100416 || 147607 || 220766 || 220254 || 99381 || 147636 || IT-4032
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || HUS726060ALA640 || 6Gb/s SATA || 7,200 || 184203 || 184153 || 87583 || 90935 || 183946 || 183679 || 87005 || 89375 || IT-4032
 +
|}
 +
 +
== Multiple Disk Results ==
 +
{| class="wikitable sortable" cellpadding=2 style="border:1px; width:100%; font-size:50%; text-align: center;"
 +
|-
 +
! colspan="6" | System Config || colspan="4" | All Disks Combined (KB/s)
 +
|-
 +
! Disk Enclosure || Controller || Disk Model || Disk Type || Spindle Speed || Qty || Read || Initial Write || Rand. Read || Rand. Write || Tender
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || ST6000NM0024 || 6Gb/s SATA || 7,200 || 16 || 1,872,830 || 1,699,285 || 1,321,172 || 1,743,663 || IT-4032
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || HUS726060ALA640 (FW 1EN) || 6Gb/s SATA || 7,200 || 16 || 1,894,552 || 1,598,613 || 1,301,298 || 1,370,605 || IT-4032
 +
|-
 +
| Promise VTRAK J630S || LSI 9207-8e || HUS726060ALA640 (FW 1EQ) || 6Gb/s SATA || 7,200 || 16 || 2,082,375 || 1,755,122 || 1,332,840 || 1,393,900|| IT-4032
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || WD4000F9YZ (FW AA2) || 6Gb/s SATA || 7,200 || 24 || 1,918,817 || 1,659,493 || 1,282,772 || 1,390,060|| IT-4032
 +
|-
 +
| Supermicro #### || LSI 9207-8e (2 Ports Connected) || WD6005FRPZ (FW ###) || 6Gb/s SATA || 7,200 || 24 || 4,025,574 || 3,761,571 || N/A || N/A || IT-4138
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || HUS726060ALA640) (FW 1EQ) || 6Gb/s SATA || 7,200 || 24 || 2,008,925 || 1,704,049 || N/A  || N/A|| IT-4138
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || HUS726060ALE610 (FW 517)|| 6Gb/s SATA || 7,200 || 24 || 2,068,481 || 1,755,754 || N/A  || N/A|| IT-4138
 +
|-
 +
| Promise VTRAK J830S || LSI 9207-8e || ### || 6Gb/s SATA || 7,200 || 24 || 0 || 0 || N/A  || N/A|| IT-4138
 +
|-
 +
| Supermicro 2028R-NR48N || N/A || Intel P3500 || NVMe || N/A || 48 || 25,622,050 || 27,792,330 || 25,639,068 || 27,782,964 || Just for fun!
 +
|}

Latest revision as of 13:12, 23 September 2016

Prerequisites

XFS & lsscsi need installing: 

yum install xfsprogs.x86_64 lsscsi

As does IOZONE: 

yum install iozone

Preparing the data filesystems

All available block devices in the attached disk enclosure will be used for this test. None of the drives in the front-end server will be used.

Get the list of available block devices

Use lsscsi to identify the drives to be tested (in this case, /dev/sdc to /dev/sdr): 

[root@nodea ~]# lsscsi
[0:0:0:0]    enclosu Promise  J630s            060=  -       
[0:0:1:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdc 
[0:0:2:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdd 
[0:0:3:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sde 
[0:0:4:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdf 
[0:0:5:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdg 
[0:0:6:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdh 
[0:0:7:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdi 
[0:0:8:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdj 
[0:0:9:0]    disk    ATA      ST6000NM0024-1HT SN02  /dev/sdk 
[0:0:10:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdl 
[0:0:11:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdm 
[0:0:12:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdn 
[0:0:13:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdo 
[0:0:14:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdp 
[0:0:15:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdq 
[0:0:16:0]   disk    ATA      ST6000NM0024-1HT SN02  /dev/sdr 
[1:0:0:0]    disk    ATA      Hitachi HUA72302 MK7O  /dev/sda 
[2:0:0:0]    disk    ATA      Hitachi HUA72302 MK7O  /dev/sdb

Create an XFS filesystem on each drive

The basic command for this is: 

mkfs.xfs -l version=2 -i size=1024 -n size=16384 -d su=64k,sw=1 <block device>

So can be scripted as follows: 

for i in {c..r};
do
	mkfs.xfs -l version=2 -i size=1024 -n size=16384 -d su=64k,sw=1 /dev/sd${i} &
done

Create mount points

Create mount points in /srv/castor/XY where XY ranges from 1 to the number of available block devices. For example, it will be 16 on a 16-bay disk enclosure.
This can be scripted as follows: 

mkdir /srv/castor
for i in {1..16};
do
	mkdir /srv/castor/${i}
done

Mount filesystems to mount points

Mount these filesystems using the following options: logbufs=8,logbsize=256k,noatime,swallow,inode64
This can be scripted to add mount point to /etc/fstab as follows: 

n=1

for i in {c..r}
do
	echo "/dev/sd${i}	/srv/castor/${n}	xfs logbufs=8,logbsize=256k,noatime,swalloc,inode64" >> /etc/fstab
	((n++))
done

Benchmarking a single disk

Cern ask for the first and last disks in the storage array to be tested and for each test to be run 4 times and an average be taken.

This benchmark can be scripted as follows: 

for j in `seq 1 4`;
do
	for i in {1,16}; # << Change 16 to the total number of drives being tested
	do
		/opt/iozone/bin/iozone -i 0 -i 1 -i 2 -R -I -l 1 -u 1 -r 1m -s 4g -F /srv/castor/"$i"/iozone.dat | tee -a iozone-1m-4g-single-disk-"$i"-loop-"$j".txt
	done
done

And then averages calculated: 

echo " "
echo "*** Results in KB/s ***"
echo " "
echo "1st Disk"
echo "=========="
echo " "
grep '  Read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1 
grep 'Initial write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1 
grep 'Random read'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1 
grep 'Random write'  iozone-1m-4g-single-disk-1-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1 
echo " "
echo "16th Disk"
echo "=========="
echo " "
grep '  Read'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1
grep 'Initial write'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1
grep 'Random read'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1
grep 'Random write'  iozone-1m-4g-single-disk-16-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1

Benchmarking all disks

Cern also ask for all disks in the storage array to be tested together and, again, for each test to be run 4 times and an average be taken.

This can be scripted as follows (in this case, with 24 drives being tested): 

for i in `seq 1 4`
do
	/opt/iozone/bin/iozone -R -I -l 24 -u 24 -r 1m -s 4g -F /srv/castor/{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24}/iozone.dat | tee -a iozone-1m-4g-24fs-loop-"$i".txt
done

For a different total number of drives, adjust the -l 24 and -u 24 values accordingly.

And then averages calculated: 

echo " "
echo "*** Results in KB/s ***"
echo " "
grep '  Read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$4; avg=sum/NR; printf "Average Read: %7.2f\n", avg}' | tail -n 1 
grep 'Initial write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Initial write: %7.2f\n", avg}' | tail -n 1 
grep 'Random read'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random read: %7.2f\n", avg}'  | tail -n 1 
grep 'Random write'  iozone-1m-4g-24fs-loop* | awk '{sum=sum+$5; avg=sum/NR; printf "Average Random write: %7.2f\n", avg}'  | tail -n 1

Single Disk Results

System Config 1st Disk (KB/s) Last Disk (KB/s)
Disk Enclosure Controller Disk Model Disk Type Spindle Speed Read Initial Write Rand. Read Rand. Write Read Initial Write Rand. Read Rand. Write Tender
Promise VTRAK J630S LSI 9207-8e ST6000NM0024 6Gb/s SATA 7,200 220139 212904 100416 147607 220766 220254 99381 147636 IT-4032
Promise VTRAK J630S LSI 9207-8e HUS726060ALA640 6Gb/s SATA 7,200 184203 184153 87583 90935 183946 183679 87005 89375 IT-4032

Multiple Disk Results

System Config All Disks Combined (KB/s)
Disk Enclosure Controller Disk Model Disk Type Spindle Speed Qty Read Initial Write Rand. Read Rand. Write Tender
Promise VTRAK J630S LSI 9207-8e ST6000NM0024 6Gb/s SATA 7,200 16 1,872,830 1,699,285 1,321,172 1,743,663 IT-4032
Promise VTRAK J630S LSI 9207-8e HUS726060ALA640 (FW 1EN) 6Gb/s SATA 7,200 16 1,894,552 1,598,613 1,301,298 1,370,605 IT-4032
Promise VTRAK J630S LSI 9207-8e HUS726060ALA640 (FW 1EQ) 6Gb/s SATA 7,200 16 2,082,375 1,755,122 1,332,840 1,393,900 IT-4032
Promise VTRAK J830S LSI 9207-8e WD4000F9YZ (FW AA2) 6Gb/s SATA 7,200 24 1,918,817 1,659,493 1,282,772 1,390,060 IT-4032
Supermicro #### LSI 9207-8e (2 Ports Connected) WD6005FRPZ (FW ###) 6Gb/s SATA 7,200 24 4,025,574 3,761,571 N/A N/A IT-4138
Promise VTRAK J830S LSI 9207-8e HUS726060ALA640) (FW 1EQ) 6Gb/s SATA 7,200 24 2,008,925 1,704,049 N/A N/A IT-4138
Promise VTRAK J830S LSI 9207-8e HUS726060ALE610 (FW 517) 6Gb/s SATA 7,200 24 2,068,481 1,755,754 N/A N/A IT-4138
Promise VTRAK J830S LSI 9207-8e ### 6Gb/s SATA 7,200 24 0 0 N/A N/A IT-4138
Supermicro 2028R-NR48N N/A Intel P3500 NVMe N/A 48 25,622,050 27,792,330 25,639,068 27,782,964 Just for fun!
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=Benchmarking:_IOZone_(CERN)&oldid=16881"