Openstack:Setting up Mapr 5.0.0 CentOS Sahara cluster - Revision history

Vipul: Added filed Sahara bug link from launchpad

2016-08-03T10:34:48Z

Added filed Sahara bug link from launchpad

Vipul at 06:39, 28 July 2016

2016-07-28T06:39:02Z

Vipul at 06:28, 28 July 2016

2016-07-28T06:28:18Z

Vipul: Created page with " Following is a description of the steps followed to setup Sahara Mapr 5.0.0 CentOS cluster on Openstack Liberty. Some steps in between include modifications done specifically..."

2016-07-28T06:12:00Z

Created page with " Following is a description of the steps followed to setup Sahara Mapr 5.0.0 CentOS cluster on Openstack Liberty. Some steps in between include modifications done specifically..."

New page

Following is a description of the steps followed to setup Sahara Mapr 5.0.0 CentOS cluster on Openstack Liberty. Some steps in between include modifications done specifically for deployment on Keele University Openstack.

Upload Mapr Image to Glance

Some pre-built sahara images are available at http://sahara-files.mirantis.com/images/upstream/liberty/. Since Sahara deployments in Liberty don't offer cloud-init, hence we need to modify the qcow2 image before uploading it to glance. For this, we use a tool called guestfish. Following changes are done to the image:

* (Keele University infra only) Add proxy for yum package manager and shell sessions. Also, add custom DHCP servers
<syntaxhighlight>
$guestfish --rw -a sahara-liberty-mapr-5.0.0-centos-6.6.qcow2
<fs> run
<fs> mount /dev/sda1 /
<fs> vi /etc/yum.conf
# Add the below line
proxy=http://wwwcache.keele.ac.uk:8080
</syntaxhighlight>

@@ Line 20: / Line 20: @@
 In the Mapr plugin inside Openstack Liberty 3.0.2, some oozie process related bugs were encountered, specifically
-* When the mapr packages are installed after spawning instances, the warden.oozie.conf is not placed in the correct directory, hence the warden process fails to recognize the presence of oozie, which ultimately make the cluster go into Error state. To overcome this, we'll be placing the warden.oozie.conf file in the correct location.
+* When the mapr packages are installed after spawning instances, the warden.oozie.conf is not placed in the correct directory, hence the warden process fails to recognize the presence of oozie, which ultimately make the cluster go into Error state. To overcome this, we'll be placing the warden.oozie.conf file in the correct location. For the same, a bug was filed in Sahara project on Launchpad. More details here : https://bugs.launchpad.net/bugs/1607704
 <syntaxhighlight>

@@ Line 18: / Line 18: @@
 </syntaxhighlight>
-* In the Mapr plugin inside Openstack Liberty 3.0.2, some oozie process related bugs were encountered, specifically
+In the Mapr plugin inside Openstack Liberty 3.0.2, some oozie process related bugs were encountered, specifically
-) When the mapr packages are installed after spawning instances, the warden.oozie.conf is not placed in the correct directory, hence the warden process fails to recognize the presence of oozie, which ultimately make the cluster go into Error state. To overcome this, we'll be placing the warden.oozie.conf file in the correct location.
 <syntaxhighlight>
 <fs> mkdir -p /opt/mapr/conf/conf.d
 <fs> write /opt/mapr/conf/conf.d/warden.oozie.conf ""
@@ Line 40: / Line 40: @@
 service.process.type=JAVA
 service.env="MAPR_MAPREDUCE_MODE=default"
 </syntaxhighlight>
-) At the end of configuration phase, sahara restarts oozie. Before going to Running state, it is stuck in standby state for some time. Sahara process only waits 60s for this to happen. Even in a big instance, oozie may not start within 60s. Hence, predefined parameter timeout's value in function _wait_for_status is changed from 60 to 180 in file /usr/lib/python2.7/site-packages/sahara/plugins/mapr/domain/node_process.py
+* At the end of configuration phase, sahara restarts oozie. Before going to Running state, it is stuck in standby state for some time. Sahara process only waits 60s for this to happen. Even in a big instance, oozie may not start within 60s. Hence, predefined parameter timeout's value in function _wait_for_status is changed from 60 to 180 in file /usr/lib/python2.7/site-packages/sahara/plugins/mapr/domain/node_process.py
 When the image is ready, upload it to glance.
@@ Line 51: / Line 49: @@
 In Data processing tab in Horizon dahsboard, register the image uploaded to glance with plugin tags 'mapr' & '5.0.0.mrv2'. For centos images, configure user as 'cloud-user'. For ubuntu images, the user would be 'ubuntu'

@@ Line 2: / Line 2: @@
 Following is a description of the steps followed to setup Sahara Mapr 5.0.0 CentOS cluster on Openstack Liberty. Some steps in between include modifications done specifically for deployment on Keele University Openstack.
-Upload Mapr Image to Glance
+== Upload Mapr Image to Glance ==
 Some pre-built sahara images are available at http://sahara-files.mirantis.com/images/upstream/liberty/. Since Sahara deployments in Liberty don't offer cloud-init, hence we need to modify the qcow2 image before uploading it to glance. For this, we use a tool called guestfish. Following changes are done to the  image:
@@ Line 8: / Line 8: @@
 * (Keele University infra only) Add proxy for yum package manager and shell sessions. Also, add custom DHCP servers
 <syntaxhighlight>
-$guestfish --rw -a sahara-liberty-mapr-5.0.0-centos-6.6.qcow2
+$ guestfish --rw -a sahara-liberty-mapr-5.0.0-centos-6.6.qcow2
 <fs> run
 <fs> mount /dev/sda1 /
@@ Line 14: / Line 14: @@
 # Add the below line
 proxy=http://wwwcache.keele.ac.uk:8080
 </syntaxhighlight>