OpenHPC:OpenStack OS Image

Prepare a filesystem CHROOT image

• Create /usr/libexec/warewulf/wwmkchroot/centos7.1-kilo.tmpl to point to updated repo data

[root@head setup-filesystems]# cat /usr/libexec/warewulf/wwmkchroot/centos7.1-kilo.tmpl
#DESC: A clone of Red Hat Enterprise Linux 7

# The general RHEL include has all of the necessary functions, but requires
# some basic variables specific to each chroot type to be defined.
. include-rhel



# Define the location of the YUM repository
YUM_MIRROR="http://mirror.centos.org/centos-7/7.2.1511/os/\$basearch/"

# Install only what is necessary/specific for this distribution
PKGLIST="SysVinit basesystem bash redhat-release chkconfig coreutils e2fsprogs \
    ethtool filesystem findutils gawk grep initscripts iproute iputils \
    mingetty mktemp net-tools nfs-utils pam portmap procps psmisc rdate rsync \
    sed setup shadow-utils rsyslog tcp_wrappers tzdata util-linux words zlib \
    tar less gzip which util-linux module-init-tools udev openssh-clients \
    openssh-server dhclient pciutils vim-minimal shadow-utils strace cronie \
    crontabs cpuspeed cpufrequtils cpio wget yum numactl libicu"




# vim:filetype=sh:syntax=sh:expandtab:ts=4:sw=4:

• Add CHROOTKILO=/opt/ohpc/admin/images/centos7.1-kilo to ~/.bashrc

[root@head images]# cat ~/.bashrc 
# .bashrc

# User specific aliases and functions

alias rm='rm -i'
alias cp='cp -i'
alias mv='mv -i'
alias l='ls -l'
alias ipmit='ipmitool -U ADMIN -P ADMIN -I lanplus -H'

CHROOTKILO=/opt/ohpc/admin/images/centos7.1-kilo
CHROOT=/opt/ohpc/admin/images/centos7.1
ohpc_repo=http://build.openhpc.community/OpenHPC:/1.0/CentOS_7.1/OpenHPC:1.0.repo
sms_name=head.vscaler.local			# Hostanem of Headnode
sms_ip=172.30.0.1				# Private Interface IP of Headnode
sms_eth_internal=enp1s0f0			# Private Interface of Headnode
eth_provision=enp1s0f0				# Provisioning Interface of Headnode
internal_netmask=255.255.0.0		# Netmaks of Private Interface
ntp_server=0.centos.pool.ntp.org		# Some NTP Server
bmc_username=ADMIN
bmc_password=ADMIN
sms_ipoib=10.10.20.1				# IPoIB Address of Headnode
ipoib_netmask=255.255.255.0			# IPoIB Netmask of Headnode

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi

• Make Initial VNFS (Chroot, compute Node template) and install some Base components into the chroot operating system

wwmkchroot centos-7 $CHROOTKILO
yum -y --installroot=$CHROOTKILO groupinstall Base
yum -y --installroot=$CHROOTKILO install kernel* grub* sudo ipmitool* epel-release htop nano tk* tcl* tigervnc* ipmitool* freeipmi* cairo* perl* gcc* glibc* screen yum-utils vim libnl 
lsof libxml2-python python mlocate numactl* lmod-ohpc ohpc-slurm-client lmod-ohpc ganglia-gmond-ohpc enviroment-modules hwlock-libs libfabric libpsm2 intel-clck-ohpc
yum -y --installroot=$CHROOTKILO remove perl-Test-mysqld-0.17-3.el7.noarch

• Setup SSH Keys for the Cluster – this is required for the root user only, /home will be exported so user ssh keys will be available.

wwinit ssh_keys
cat ~/.ssh/cluster.pub >> $CHROOTKILO/root/.ssh/authorized_keys

• Add mount points to /etc/fstab

echo "${sms_ip}:/home /home nfs nfsvers=3,rsize=1024,wsize=1024,cto 0 0" >> $CHROOT/etc/fstab
echo "${sms_ip}:/opt/ohpc/pub /opt/ohpc/pub nfs nfsvers=3,rsize=1024,wsize=1024,cto 0 0" >> $CHROOT/etc/fstab

• Copy over resolv.conf to the chroot and modify the contents to point to the headnode and google dns.

cp /etc/resolv.conf $CHROOTKILO/etc/
vi $CHROOTKILO/etc/resolv.conf

• Modify Limits to unlimited on headnode and compute nodes..

echo "* soft memlock unlimited" >> $CHROOTKILO/etc/security/limits.conf
echo "* hard memlock unlimited" >> $CHROOTKILO/etc/security/limits.conf

• Building the bootstrap and vnfs images

[root@head ~]# wwbootstrap 3.10.0-327.10.1.el7.x86_64
Number of drivers included in bootstrap: 433
Number of firmware images included in bootstrap: 93
Building and compressing bootstrap
Integrating the Warewulf bootstrap: 3.10.0-327.10.1.el7.x86_64
Including capability: provision-adhoc
Including capability: provision-files
Including capability: provision-selinux
Including capability: provision-vnfs
Including capability: setup-filesystems
Including capability: transport-http
Compressing the initramfs
Locating the kernel object
Bootstrap image '3.10.0-327.10.1.el7.x86_64' is ready
Done.
[root@head ~]# wwvnfs -y --chroot $CHROOTKILO
Using 'centos7.1-kilo' as the VNFS name
Creating VNFS image from centos7.1-kilo
Building new chroot...
Building and compressing the final image
Cleaning temporary files
VNFS 'centos7.1-kilo' has been imported
Done.
Wrote a new configuration file at: /etc/warewulf/vnfs/centos7.1-kilo.conf

Mount the CHROOT filesystem locally for editing

• The above process should have created a vanilla CentOS 7.1 image which could be deployed to compute nodes. We need to chroot to the root filesystem and modify it with all the OpenStack goodness.

• Before chrooting the new filesystem, we need to mount the host's /dev pseudo-directory to the new filesystem:

[root@head ~]# mount --bind /dev $CHROOTKILO/dev

• Now go ahead and chroot to the new filesystem:

[root@head ~]# chroot $CHROOTKILO

Install OpenStack Kilo RDO release and packstack

[root@head /]# yum install -y http://rdoproject.org/repos/openstack-kilo/rdo-release-kilo.rpm
[root@head /]# yum -y install openstack-packstack

Create a packstack answer file

• Use packstack to create an answer file:

[root@head /]# packstack --gen-answer-file /root/answers.txt
Packstack changed given value  to required value /root/.ssh/id_rsa.pub
[root@head /]#

• Edit the answers.txt file with some settings:

CONFIG_NTP_SERVERS=pool.ntp.org
CONFIG_KEYSTONE_ADMIN_PW=Boston2016
CONFIG_HORIZON_SSL=y
CONFIG_PROVISION_DEMO=n

• Also make sure the IP addresses in the answer file are appropriate for the target system (by default, they will point to the head node which is not a good situation). For example:

CONFIG_CONTROLLER_HOST=172.30.1.1
CONFIG_COMPUTE_HOSTS=172.30.1.1
CONFIG_NETWORK_HOSTS=172.30.1.1
CONFIG_STORAGE_HOST=172.30.1.1
CONFIG_SAHARA_HOST=172.30.1.1
CONFIG_AMQP_HOST=172.30.1.1
CONFIG_MARIADB_HOST=172.30.1.1
CONFIG_KEYSTONE_LDAP_URL=ldap://172.30.1.1
CONFIG_MONGODB_HOST=172.30.1.1
CONFIG_REDIS_MASTER_HOST=172.30.1.1

NB: Moving forwards, we should look at making the contents of this file dynamic so OpenHPC populates the IP addresses according to the host it is going be installed on.

Rebuild the vnfs image

• As before rebuild the vnfs image: NB this process takes about 20 minutes on our test system!

[root@head ~]# wwvnfs -y --chroot $CHROOTKILO
Using 'centos7.1-kilo' as the VNFS name
Creating VNFS image from centos7.1-kilo
Building new chroot...
Building and compressing the final image
Cleaning temporary files
VNFS 'centos7.1-kilo' has been imported
Done.
Wrote a new configuration file at: /etc/warewulf/vnfs/centos7.1-kilo.conf