Excelero Nvmesh

Prerequisites

Excelero requires certain OS and kernel versions to be supported. As of 11/04/2018.

May want to look at using PDSH to allow for remote execution across all nodes at once, safes time. If using GPFS it has a built in tool called mmdsh.

For example

[root@excelero-a tmp]# pdsh -w excelero-a,excelero-b,excelero-c,excelero-d uptime
excelero-a:  11:57:24 up 4 days, 19:25,  4 users,  load average: 0.21, 0.11, 0.14
excelero-d:  11:57:24 up 4 days, 19:24,  1 user,  load average: 0.18, 0.15, 0.22
excelero-b:  11:57:24 up 4 days, 19:25,  1 user,  load average: 0.33, 0.32, 0.30
excelero-c:  11:57:24 up 4 days, 19:25,  1 user,  load average: 0.08, 0.03, 0.13

Centos

7.3

Tried and tested (10/04/2018): 3.10.0-514.el7.x86_64. Distro ISO: CentOS-7-x86_64-Minimal-1611

You will need Headers and Devel packages during the install however the version on repo did not match the kernel installed,

\\10.0.0.222\software\Excelero\kernel-headers-3.10.0-514.el7.x86_64.rpm

\\10.0.0.222\software\Excelero\kernel-devel-3.10.0-514.el7.x86_64.rpm

7.4

OFED and Kernels that can be used in 7.4

MLNX_OFED_LINUX-4.2-1.0.0.0-rhel7.4-x86_64.tgz

"3.10.0-693.5.2.el7.x86_64"

"3.10.0-693.11.1.el7.x86_64"

MLNX_OFED_LINUX-4.1-1.0.2.0-rhel7.4-x86_64.tgz

"3.10.0-693.5.2.el7.x86_64"

"3.10.0-693.el7.x86_64"

No OFED

• 0 : "3.10.0-693.11.1.el7.x86_64"

• 1 : "3.10.0-693.5.2.el7.x86_64"

Ubuntu

Version: Ubuntu 16.04.3

Kernels supported: 4.4.0-103-generic, 4.4.0-108-generic and 4.4.0-116-generic.

NVmesh packages/ repo

We had a mixed experience since their repo is really slow (45kb/s). So i would just manually install by the packages listed below or create you own repo onsite.

Centos/RHEL 1.2.1-217 =

\\10.0.0.222\software\Excelero\NVMesh-target-1.2.1-217.x86_64.rpm \\10.0.0.222\software\Excelero\NVMesh-client-1.2.1-217.x86_64.rpm

Centos/RHEL 1.2.1-194 =

\\10.0.0.222\software\Excelero\NVMesh-target-1.2.1-194.x86_64.rpm \\10.0.0.222\software\Excelero\NVMesh-client-1.2.1-194.x86_64.rpm

Installation of NVmesh onto Centos 7

Tried and tested

Chose which node on you`re chosen infrastructure to install the NVmesh management GUI.

To install the web based administration GUI you need to install Mongo DB and nodejs 6.* or 8.*

Create a file called mongo.repo and insert the content below

vim /etc/yum.repos.d/mongo.repo

Here is a example repo entry for mongodb:

[mongodb-org-3.4]
name=MongoDB Repository
baseurl=http://repo.mongodb.org/yum/redhat/7/mongodb-org/3.4/x86_64/
gpgcheck=0
enabled=1

To install Mongodb:

yum install mongodb-org -y

Install epel repo and nodejs:

yum install epel-release
yum install nodejs

However when we tried to install this, we found that the epel-release of nodejs would install correctly but nvmesh management would break and proceed to never start. Using the alternative way gets around this, its not a error per say but the nvmeshmanagement service will just never start with nodejs from epel. Alternative way: Add nodejs repo:

curl --silent --location https://rpm.nodesource.com/setup_6.x | bash -

Install nodejs:

yum install nodejs -y

3. Verify mongodb is running service mongod status

4. Open port 4000 and 4001. 4001 is only required when having more than one Management server.

sudo iptables -I INPUT 1 -m state --state NEW -m tcp -p tcp --dport 4000 -j ACCEPT -m comment --comment Excelero-Management
sudo firewall-cmd --permanent --direct --add-rule ipv4 filter INPUT 0 -p -tcp --dport 4000 -j ACCEPT -m comment --comment Excelero-Management

Alternatively just disable firewalld completely if this is not important to you.

5. Install NVMesh from the repo below -

[NVMesh] 
name=NVMesh repository 
baseurl=https://bostonuk:bostonuk@repo.excelero.com/repos/NVMesh/redhat/7.3
gpgcheck=0 
enabled=1
 

or install the packages listed above.

Install management first and check functionality.

yum install NVMesh-management-1.2.1-56.x86_64.rpm -y

Go to in browser: https://172.28.0.181:4000/ and this should resolve the NVMesh GUI, if at this stage you get nothing. Check that the nvmeshmanagement service is running, check that the ports are cleared or shutoff firewalld. If the service is up but showing errors then this is most likely a nodejs issue. If you installed nodejs from EPEL then remove this and install with the alterative method.

Default login details

login: admin@excelero.com pass: admin

Error creating thumbnail: File missing

Error creating thumbnail: File missing

Once the management interface is up and working you will need to install at least the target package on the nodes you wish to use as storage nodes.

Install the client RPM onto the server/node/workstation that you want to have access to NVmesh.

In our testing our nodes were hyperconverged. IE both clients and servers.

So we installed both packages:

yum install NVMesh-target-1.2.1-217.x86_64.rpm -y
yum install NVMesh-client-1.2.1-217.x86_64.rpm -y

To get all the nodes working

6. Restart nvmeshmgr service.

7. Install NVMesh client and target RPMs (note: the Client RPMs contains the common module so get prompted to install this dependency even if you dont want to run the Client service. We will change this in a later release. It is just “cosmetic” the time being).

Lastly, run nvmesh_format.py to format the disks you plan to use.