Lustre SFF

Introduction

Lustre SFF (Small Form Factor), is a compact deployment of ZFS-backed (Zettabyte File System) Lustre intended as an alternative to NFS for a comparable capacity and scalability.

http://lustre.ornl.gov/ecosystem-2016/documents/tutorials/Stearman-LLNL-ZFS.pdf

Intel Enterprise Edition for Lustre White Paper

The Intel January 2014 white paper "Architecting a high performance storage system" serves as a good starting point for optimizing Lustre SFF.

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/architecting-lustre-storage-white-paper.pdf

Backend Storage

smartctl

smartctl (smartmontools; Self-Monitoring, Analysis and Reporting Technology System) is used to uniquely identify devices, conduct device testing, and assess the health of devices.

sgpdd_survey

sgpdd-survey (sg3_utils{,-libs} and Lustre iokit https://downloads.hpdd.intel.com/public/lustre ) is used to analyze backend storage (dd is not suitable as response to multiple IO threads is of interest).

rszlo-rszhi: record size in KB; Affects how many blocks can be transferred in each transaction. Simulates Lustre RPC size.
crglo-crghi: number of regions; Simulates multiple Lustre clients per OST. More regions requires more seeking and hence lower performance.
thrlo-thrhi: number of threads; Simulates OSS threads.
size: total size in MB
blocksize: 512 B; Default size is 8 GB and blocksize is 512 B but 32 GB (or 2x system memory) and 1 MB blocksize recommended to simulate Lustre sequential workload.

Recommended parameters: rszhi=1024, thrhi=16, crghi=16, size=32768 (or twice RAM), dio=1, oflag=direct, iflag=direct bs=1048576

obdfilter-survey

case: local-disk, network-echo, network-disk; Run survey on disk-backed local obdfilter instances, network loopback or disk instances.
thrlo-thrhi: Number of threads
nobjlo-nobjhi: Number of objects to read/write.
rszlo-rszhi: Record size in KB.
size: Total IO size in MB.
targets: Names of obdfilter instances.

Recommended parameters: rszlo=rszhi=1024, nobjhi=128, thrhi=128

http://wiki.lustre.org/images/4/40/Wednesday_shpc-2009-benchmarking.pdf

Installing ZFS