OpenStack: Hadoop Cluster Setup on Sahara

Steps to setup Hadoop cluster with Sahar in Openstack Liberty OS : CentOS 7

• Setup Openstack Liberty

$ sudo yum install -y centos-release-openstack-liberty
$ sudo yum update -y
$ sudo yum install -y openstack-packstack
$ packstack --allinone

This installs Openstack Liberty without Sahara

After running packstack for the first time, a packstack-answers file will be generated which contains the overall Openstack config. Now, edit the latest answers file and enable Sahara by editing the line

CONFIG_HEAT_INSTALL=n

to

CONFIG_HEAT_INSTALL=y

Now run packstack again

$ packstack --answer-file packstack-answers-<timestamp>.txt

This will install Openstack Liberty with Sahara enabled.

• Configure Sahara to deploy a Vanilla Apache Hadoop cluster

• Upload and register an image

1) Upload a vanilla Hadoop image to Glance through Images section in Compute tab present in the Horizon dashboard. Prepared isos can be availed from http://sahara-files.mirantis.com/images/upstream/liberty/

2) In the data processing tab under Image Registry section, register the already uploaded image in Glance. Also, add tags to the image corresponding to the Hadoop version which it belongs to. For Example: Add tag 'vanilla' and '2.7.1' if you want to use that image to create a Vanilla Hadoop 2.7.1 cluster. Note: Vanilla 2.6.0 is deprecated and not supported by Openstack.

• Create node group template

1) Create a node group template under Data Processing tab. Select the desired hadoop plugin name and version.

2) Specify the configuration for nodes which you want to spawn in the cluster, including flavor of machine and which hadoop processes you want to run.

• Create Cluster Template

1) Create a cluster template under Data Processing tab. Select the desired hadoop plugin name and version.

2) Describe template config including the node group template that you created in the previous step.

3) Specify any other HDFS, Hadoop and MapReduce parameters that you want to use.

• Launch Cluster

1) Launch a cluster under Data Processing tab. Select the desired hadoop plugin name and version.

2) Select desired cluster template created in the previous step and the count of clusters to be created. Choose base vanilla hadoop image that was uploaded and registered earlier, alongwith keypair and Neutron network to launch the cluster into.

3) Sahara will now spawn the instances which would be visible inside Instances section in the Compute tab. When these instances are ready, they can be accessed through console in Horizon or ssh login with provided keypair.