Difference between revisions of "Lustre SFF"
(sgpdd_survey params) |
(obdfilter-survey and details.) |
||
| Line 17: | Line 17: | ||
==== sgpdd_survey ==== | ==== sgpdd_survey ==== | ||
| − | + | sgpdd-survey (sg3_utils) is used to analyze backend storage (dd is not suitable as response to multiple IO threads is of interest). | |
; rszlo-rszhi: record size in KB | ; rszlo-rszhi: record size in KB | ||
| Line 28: | Line 28: | ||
; blocksize: 512 B | ; blocksize: 512 B | ||
: Default size is 8 GB and blocksize is 512 B but 32 GB (or 2x system memory) and 1 MB blocksize recommended to simulate Lustre sequential workload. | : Default size is 8 GB and blocksize is 512 B but 32 GB (or 2x system memory) and 1 MB blocksize recommended to simulate Lustre sequential workload. | ||
| + | |||
| + | Recommended parameters: rszhi=1024, thrhi=16, crghi=16, size=32768 (or twice RAM), dio=1, oflag=direct, iflag=direct bs=1048576 | ||
| + | |||
| + | ==== obdfilter-survey ==== | ||
| + | |||
| + | ; case: local-disk, network-echo, network-disk | ||
| + | : Run survey on disk-backed local obdfilter instances, network loopback or disk instances. | ||
| + | ; thrlo-thrhi | ||
| + | : Number of threads | ||
| + | ; nobjlo-nobjhi | ||
| + | : Number of objects to read/write. | ||
| + | ; rszlo-rszhi | ||
| + | : Record size in KB. | ||
| + | ; size | ||
| + | : Total IO size in MB. | ||
| + | ; targets | ||
| + | : Names of obdfilter instances. | ||
| + | |||
| + | Recommended parameters: rszlo=rszhi=1024, nobjhi=128, thrhi=128 | ||
http://wiki.lustre.org/images/4/40/Wednesday_shpc-2009-benchmarking.pdf | http://wiki.lustre.org/images/4/40/Wednesday_shpc-2009-benchmarking.pdf | ||
Revision as of 09:29, 14 October 2016
Introduction
Lustre SFF (Small Form Factor), is a compact deployment of ZFS-backed (Zettabyte File System) Lustre intended as an alternative to NFS for a comparable capacity and scalability.
http://lustre.ornl.gov/ecosystem-2016/documents/tutorials/Stearman-LLNL-ZFS.pdf
Intel Enterprise Edition for Lustre White Paper
The Intel January 2014 white paper "Architecting a high performance storage system" serves as a good starting point for optimizing Lustre SFF.
Backend Storage
smartctl
smartctl (smartmontools; Self-Monitoring, Analysis and Reporting Technology System) is used to uniquely identify devices, conduct device testing, and assess the health of devices.
sgpdd_survey
sgpdd-survey (sg3_utils) is used to analyze backend storage (dd is not suitable as response to multiple IO threads is of interest).
- rszlo-rszhi
- record size in KB
- Affects how many blocks can be transferred in each transaction. Simulates Lustre RPC size.
- crglo-crghi
- number of regions
- Simulates multiple Lustre clients per OST. More regions requires more seeking and hence lower performance.
- thrlo-thrhi
- number of threads
- Simulates OSS threads.
- size
- total size in MB
- blocksize
- 512 B
- Default size is 8 GB and blocksize is 512 B but 32 GB (or 2x system memory) and 1 MB blocksize recommended to simulate Lustre sequential workload.
Recommended parameters: rszhi=1024, thrhi=16, crghi=16, size=32768 (or twice RAM), dio=1, oflag=direct, iflag=direct bs=1048576
obdfilter-survey
- case
- local-disk, network-echo, network-disk
- Run survey on disk-backed local obdfilter instances, network loopback or disk instances.
- thrlo-thrhi
- Number of threads
- nobjlo-nobjhi
- Number of objects to read/write.
- rszlo-rszhi
- Record size in KB.
- size
- Total IO size in MB.
- targets
- Names of obdfilter instances.
Recommended parameters: rszlo=rszhi=1024, nobjhi=128, thrhi=128
http://wiki.lustre.org/images/4/40/Wednesday_shpc-2009-benchmarking.pdf