Quobyte
Server Installation
Before installing Quobyte server software there are some tasks that need to be performed on each server to ensure correct functionality.
Configure NTP
Each server needs to have the same time, or some services will not start. Ensure that NTP is configured and running on all servers, and check all clocks are synced.
Disable Swap
Disable swap on each storage server. Running swapoff -a will disable all swap devices found in /proc/swaps and /etc/fstab. Also comment out/remove any swap lines in /etc/fstab to prevent swap being activated if a server is rebooted.
Install dependencies
yum -y install java-1.8.0-openjdk-headless wgetDownload the Quobyte yum repo file
cd /etc/yum.repos.d
wget https://packages.quobyte.com/repo/9/<YOUR_REPO_ID>/rpm/CentOS_7/quobyte.repoInstall Quobyte packages
yum -y install quobyte-server quobyte-clientServer Configuration
Prepare Drives
Any drives being used by Quobyte need to be formatted and mounted before Quobyte can use them. Currently only ext4 and XFS are supported. Each server in our testbed has 3 available drives. 2 SSDs (/dev/sdb and /dev/sdc) and 1 HHD (/dev/sdd). To prepare each drive do the following
# Create a filesystem on each drive and mount them.
# Note it is recommended to use the full drive and not partitions.
mkfs.xfs /dev/sdX
mount /dev/sdX /some/mount/point
# The testbed was configured as per below, where /dev/sda was the OS drive
[root@q01 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 238.5G 0 disk
├─sda1 8:1 0 1M 0 part
├─sda2 8:2 0 512M 0 part /boot
├─sda3 8:3 0 15.6G 0 part
├─sda4 8:4 0 1K 0 part
└─sda5 8:5 0 222.4G 0 part /
sdb 8:16 0 372.6G 0 disk /mnt/quobyte/metadata0
sdc 8:32 0 894.3G 0 disk /mnt/quobyte/data0
sdd 8:48 0 931.5G 0 disk /mnt/quobyte/data1
# The same procedure was performed on each storage serverDefine Registry Servers
Edit /etc/quobyte/host.cfg on each server to state which servers are running the registry servers. In our setup all 4 are so it was updated to read
registry=q01:7861,q02:7861,q03:7861,q04:7861
If name resolution isn't configured on the servers the hostnames can be replaced with IP addresses.
Create registry devices
To create the first registry device do the following on one server only.
qbootstrap /mnt/quobyte/metadata0Then start services on this server.
systemctl start quobyte-registry
systemctl start quobyte-webconsole
systemctl start quobyte-apiTo confirm that the registry services is running and the device is available run the following command.
[root@q01 ~]# qmgmt device list
Id Host Mode Disk Used Disk Avail Services LED Mode
1 q01 ONLINE 34 MB 400 GB REGISTRY OFFNote it make take a minute for the device to initially register.
To create other registry services, do the following on each server
qmkdev -t REGISTRY /mnt/quobyte/metadata0
systemctl start quobyte-registryOnce this is completed on each server you can list and check availability of each registry from the first server.
[root@q01 ~]# qmgmt device list
Id Host Mode Disk Used Disk Avail Services LED Mode
1 q01 ONLINE 34 MB 400 GB REGISTRY OFF
2 q02 ONLINE 34 MB 400 GB REGISTRY OFF
3 q03 ONLINE 34 MB 400 GB REGISTRY OFF
4 q04 ONLINE 34 MB 400 GB REGISTRY OFFAdd Metadata Devices
From the first server run the following command to add Metadata to the registry devices
qmgmt device update add-type <id> METADATA
# id for each registry is listed in the output of 'qmgmt device list'SSH to each host with a metadata device and start the metadata service by running
systemctl start quobyte-metadataConfirm that metadata devices are running
[root@q01 ~]# qmgmt device list
Id Host Mode Disk Used Disk Avail Services LED Mode
1 q01 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
2 q02 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
3 q03 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
4 q04 ONLINE 34 MB 400 GB METADATA REGISTRY OFFAdd Data Devices
To add data devices perform the following on each server.
# Define data devices
qmkdev -t DATA /mnt/quobyte/data0
qmkdev -t DATA /mnt/quobyte/data1
# Start quobyte-data service
systemctl start quobyte-dataOnce completed on each server check all devices are registered and available.
[root@q01 ~]# qmgmt device list
Id Host Mode Disk Used Disk Avail Services LED Mode
1 q01 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
5 q01 ONLINE 22 GB 960 GB DATA OFF
6 q01 ONLINE 34 GB 1000 GB DATA OFF
2 q02 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
7 q02 ONLINE 36 MB 960 GB DATA OFF
8 q02 ONLINE 46 GB 1000 GB DATA OFF
3 q03 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
9 q03 ONLINE 36 MB 960 GB DATA OFF
10 q03 ONLINE 46 GB 1000 GB DATA OFF
4 q04 ONLINE 34 MB 400 GB METADATA REGISTRY OFF
11 q04 ONLINE 36 MB 400 GB DATA OFF
12 q04 ONLINE 46 GB 1000 GB DATA OFF