Difference between revisions of "XCAT Installation on Centos 8.3"

From Define Wiki
Jump to navigation Jump to search
(Created page with "== Base Level Setup == Note: Starting point Centos minimal 8.2 with networking in place and setup We start by setting the hostname and disabling SElinux <pre> setenforce 0...")
 
Line 1: Line 1:
 
== Base Level Setup ==
 
== Base Level Setup ==
−
Note: Starting point Centos minimal 8.2 with networking in place and setup
+
As the installation instructions will require specific values such as IP Addresses, MAC Addresses and hostnames these instructions will use data substitution place holder of '''<data>'''.
−
 
 
−
We start by setting the hostname and disabling SElinux
 
−
 
 
−
<pre>
 
−
setenforce 0
 
−
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
 
−
yum -y install vim tmux wget rsyslog
 
−
systemctl enable rsyslog
 
−
systemctl start rsyslog
 
−
</pre>
 
  
 +
Build server with Base Operating Software, we will use CentOS 8.3 minimal installation.
 +
Confirm networking is configured with Static IP Address for network interface.
  
 
== Setup naming and hosts file ==
 
== Setup naming and hosts file ==
Line 17: Line 9:
  
 
<pre>
 
<pre>
−
hostnamectl set-hostname deploy.dt.internal  
+
hostnamectl set-hostname '''<server_name.domain_name>''' # e.g. deploy.dt.internal
 
</pre>
 
</pre>
  
−
Set the '''/etc/hosts''' file  
+
'''Check/Set''' the '''/etc/hosts''' file  
  
 
<pre>
 
<pre>
 
# Ensure there is a /etc/hosts entry for the internal interface
 
# Ensure there is a /etc/hosts entry for the internal interface
  
−
192.168.102.253 deploy.dt.internal      deploy
+
'''<ip_address> <server_name> <server_name.domain_name>'''
 
</pre>
 
</pre>
  
−
== Disable the firewall ==
+
== Disable the firewall and SELINUX ==
 
Disable the firewall
 
Disable the firewall
  
 
<pre>
 
<pre>
 +
systemctl stop firewalld
 
systemctl disable firewalld
 
systemctl disable firewalld
−
systemctl stop firewalld
 
 
</pre>
 
</pre>
  
 +
Disable SELINUX
 +
<pre>
 +
'setenforce 0' or 'echo 0 > /sys/fs/selinux/enforce'  # This sets the mode to permissive mode temporarily until reboot.
 +
 +
sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
 +
</pre>
 +
You need to reboot the server to make sure SELINUX is permanently disabled.
 +
 +
Check to see if SELINUX is disabled by running
 +
<pre>
 +
sestatus
 +
</pre>
 +
 +
== Enable the time service (Chronyd) ==
 +
<pre>
 +
# uncomment the following to allow the compute nodes access
 +
# Allow NTP client access from local network.
 +
allow 192.168.0.0/16
 +
 +
# add to the end of the file /etc/chrony.conf (note below for Indonesia, select correct tz)
 +
echo "
 +
server 1.id.pool.ntp.org
 +
server 2.id.pool.ntp.org
 +
server 3.id.pool.ntp.org
 +
server 4.id.pool.ntp.org" >> /etc/chrony.conf
 +
 +
systemctl enable chronyd
 +
systemctl start chronyd
 +
</pre>
 +
 +
== Install and configure some required packages ==
 +
<pre>
 +
yum -y install vim tmux wget rsyslog
 +
</pre>
  
−
== Setup the software repos ==
+
Enable and start rsyslog
 +
<pre>
 +
systemctl enable rsyslog
 +
systemctl start rsyslog
 +
</pre>
  
 +
== Setup the xCAT software repos ==
 
<pre>
 
<pre>
 
yum -y install yum-utils
 
yum -y install yum-utils
Line 47: Line 78:
 
</pre>
 
</pre>
  
−
== Add provisioning services on the headnode ==
+
== Install and provision xCAT onto management node ==
−
 
 
 
<pre>
 
<pre>
 
yum -y install xCAT  
 
yum -y install xCAT  
Line 55: Line 85:
 
</pre>
 
</pre>
  
−
== Setup Networking for provisioning ==
+
== Configure system password for the root user on the compute nodes ==
−
 
 
−
In this example we are using ```enp1s0f1``` as the internal interface (i.e. the pxeboot / provisioning interface)
 
−
 
 
 
<pre>
 
<pre>
−
# setup networking for provisioning
+
chtab key=system passwd.username=root passwd.password=`openssl rand -base64 12`
−
${sms_eth_internal}=enp1s0f1
 
−
${sms_ip}=192.168.102.253
 
−
${internal_netmask}=24
 
−
 
 
−
# these commands seemed pointless as i already had setup - but revisit if issues later on. not sure what that broadcast + command does
 
−
ip link set dev enp1s0f1 up
 
−
ip address add 192.168.102.253/24 broadcast + dev enp1s0f1
 
−
 
 
−
# set the DHCP interface
 
−
chdef -t site dhcpinterfaces="xcatmn|enp1s0f1"
 
 
</pre>
 
</pre>
 +
# Note; for the password generation command use the Grave Accent (backquote)` to enclose the command rather than the apostrophe '
  
−
== Add the Centos DVD to the installer ==
+
== Set the site domain name ==
−
 
 
−
Update the URL for more recent versions if required.
 
−
 
 
−
NOTE: don't use the minimal iso, grab the larger DVD iso
 
−
 
 
−
 
 
 
<pre>
 
<pre>
−
# grab the centos dvd iso to create a local repo
+
chdef -t site domain=dt.internal
−
wget http://mirrors.ukfast.co.uk/sites/ftp.centos.org/8.2.2004/isos/x86_64/CentOS-8.2.2004-x86_64-dvd1.iso
 
−
 
 
−
copycds ./CentOS-8.2.2004-x86_64-dvd1.iso
 
 
</pre>
 
</pre>
  
  
−
Query available images
+
== Setup Networking for provisioning ==
−
 
+
Set the DHCP interface
 
<pre>
 
<pre>
−
lsdef -t osimage
+
chdef -t site dhcpinterfaces="<server_name>|<server_ifname>"
 
</pre>
 
</pre>
  
 
== Enable ssh during the installation on the compute nodes ==
 
== Enable ssh during the installation on the compute nodes ==
 +
Enables ssh to the node during installation
 
<pre>
 
<pre>
−
# enables ssh to the node during installation
 
 
chdef -t site clustersite xcatdebugmode=2
 
chdef -t site clustersite xcatdebugmode=2
 +
lsdef -t site clustersite
 +
</pre>
 +
List parameters in clustersite
 +
<pre>
 
lsdef -t site clustersite
 
lsdef -t site clustersite
 
</pre>
 
</pre>
  
−
== Enable the time service (Chronyd) ==
+
== Add the Centos DVD to the installer ==
 +
Check the CentOS website for URL of the required version of CentOS DVD version required and then download
 +
<pre>
 +
wget http://mirrors.ukfast.co.uk/sites/ftp.centos.org/8.3.2011/isos/x86_64/CentOS-8.3.2011-x86_64-dvd1.iso
 +
</pre>
 +
Once downloaded import it into the xCAT database
 +
<pre>
 +
copycds ./CentOS-8.3.2011-x86_64-dvd1.iso
 +
</pre>
 +
Query available images
 
<pre>
 
<pre>
−
# uncomment the following to allow the compute nodes access
+
lsdef -t osimage
−
# Allow NTP client access from local network.
 
−
allow 192.168.0.0/16
 
−
 
 
−
# add to the end of the file /etc/chrony.conf (note below for Indonesia, select correct tz)
 
−
echo "
 
−
server 1.id.pool.ntp.org
 
−
server 2.id.pool.ntp.org
 
−
server 3.id.pool.ntp.org
 
−
server 4.id.pool.ntp.org" >> /etc/chrony.conf
 
−
 
 
−
systemctl enable chronyd
 
−
systemctl start chronyd
 
 
</pre>
 
</pre>
  
 
== Add a node to the installation. ==
 
== Add a node to the installation. ==
−
 
 
<pre>
 
<pre>
−
# add a node
+
mkdef -t node <node_name> groups=compute,all ip=<node_ip_address> mac=<node_mac_address> netboot=xnba arch=x86_64 bmc=<bmc_ip_address> bmcusername=<ipmi_username> bmcpassword=<ipmi_password> mgt=ipmi serialport=0 serialspeed=115200 provmethod=centos8-x86_64-install-compute
−
# note: serialport=0 (ttyS0)
 
−
#      serialport=1 (ttyS1)
 
−
[root@deploy ~]# mkdef -t node node0001 groups=compute,all ip=192.168.102.10 mac=00:15:B2:AA:E2:60 netboot=xnba arch=x86_64 bmc=192.168.101.10 bmcusername=admin bmcpassword=admin mgt=ipmi serialport=0 serialspeed=115200 provmethod=centos8-x86_64-install-compute
 
−
1 object definitions have been created or modified.
 
−
 
 
−
# set the root password
 
−
chtab key=system passwd.username=root passwd.password=`openssl rand -base64 12`
 
−
 
 
−
# set the domain name
 
−
chdef -t site domain=dt.internal
 
 
</pre>
 
</pre>
  
−
 
+
== Finalise the setup ==
−
Finalise the setup - below will create the /tftpboot files and /etc/hosts etc.
+
Complete network service configurations
 
<pre>
 
<pre>
−
# finalise the setup
 
−
# Complete network service configurations
 
−
# Please note: you will see a warning about how the network not having a dynamic range. Its safe to ignore this.
 
 
makehosts
 
makehosts
 
makenetworks  
 
makenetworks  
−
makedhcp -n
+
makedhcp -n ; Note. 'Warning: [deploy]: No dynamic range specified for 192.168.97.0. If hardware discovery is being used, a dynamic range is required.' message can be ignored as IP address is defined for the node itself.
 
makedns -n
 
makedns -n
 
</pre>
 
</pre>
−
 
−
 
  
 
== Check the rcons status ==
 
== Check the rcons status ==
 
<pre>
 
<pre>
−
# might not be needed systemctl enable goconserver
+
systemctl status goconserver
−
# systemctl start goconserver
+
</Pre>
 +
If goconserver is not enabled or running run the appropriate commands below depending on status
 +
<pre>
 +
systemctl enable goconserver
 +
systemctl start goconserver # Not necessarily required as next command will start the server if not already running
 +
</pre>
 +
Start goconserver if not started
 +
<pre>
 
makegocons
 
makegocons
 +
</pre>
 +
List the console connection of the nodes
 +
<pre>
 
makegocons -q
 
makegocons -q
 
</pre>
 
</pre>
  
 +
== Set the nodes to pxeboot ==
 +
Associate desired provisioning image for computes
 +
<pre>
 +
nodeset compute osimage=centos8-x86_64-install-compute
 +
<pre>
 +
 +
== Precheck that the xCAT management node is ready for OS provision ==
 +
<pre>
 +
xcatprobe xcatmn -i <server_ifname>
 +
</pre>
  
 +
== Install the OS to the node and then check Installation of the node ==
 +
<pre>
 +
rinstall <node_name> osimage=centos8-x86_64-install-compute # osimage is optional if it has already been defined by the provmethod of the node configuration
 +
</pre>
 +
Once the installation is complete run the command to see the OS version installed on the node.
 +
<pre>
 +
xdsh '''<node_name>''' more '''/etc/*release'''
 +
</pre>
  
−
== Set the nodes to pxeboot ==
+
== Adding additional nodes ==
 +
Define the new node with the mkdef command.
 +
<pre>
 +
mkdef -t node <node_name> groups=compute,all ip=<node_ip_address> mac=<node_mac_address> netboot=xnba arch=x86_64 bmc=<bmc_ip_address> bmcusername=<ipmi_username> bmcpassword=<ipmi_password> mgt=ipmi serialport=0 serialspeed=115200 provmethod=centos8-x86_64-install-compute
 +
</pre>
 +
Update the hosts file with the new node.
 +
<pre>
 +
makehosts <new_node_name>
 +
</pre>
 +
Update DNS with the new node.
 +
<pre>
 +
makedns <new_host_name>
 +
</pre>
 +
Install OS on the new node.
 +
<pre>
 +
rinstall <node_name> osimage=centos8-x86_64-install-compute
 +
</pre>
 +
Once the installation is complete run the command to see the OS version installed on the node.
 
<pre>
 
<pre>
−
# Associate desired provisioning image for computes
+
xdsh <new_node_name> more /etc/*release
−
nodeset compute osimage=centos8-x86_64-install-compute
 
−
rinstall nvme0001 osimage=centos8-x86_64-install-compute
 
 
</pre>
 
</pre>
  
Line 171: Line 208:
 
# check the node stats
 
# check the node stats
 
lsdef -t node -l
 
lsdef -t node -l
−
lsdef -o node0001
+
lsdef -o <node_name>
  
 
# check the operating systems stats  
 
# check the operating systems stats  

Revision as of 16:43, 22 February 2021

Base Level Setup

As the installation instructions will require specific values such as IP Addresses, MAC Addresses and hostnames these instructions will use data substitution place holder of .

Build server with Base Operating Software, we will use CentOS 8.3 minimal installation. Confirm networking is configured with Static IP Address for network interface.

Setup naming and hosts file

Set the hostname 

hostnamectl set-hostname '''<server_name.domain_name>''' # e.g. deploy.dt.internal

Check/Set the /etc/hosts file 

# Ensure there is a /etc/hosts entry for the internal interface

'''<ip_address> <server_name> <server_name.domain_name>'''

Disable the firewall and SELINUX

Disable the firewall 

systemctl stop firewalld
systemctl disable firewalld

Disable SELINUX 

'setenforce 0' or 'echo 0 > /sys/fs/selinux/enforce'  # This sets the mode to permissive mode temporarily until reboot.

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

You need to reboot the server to make sure SELINUX is permanently disabled.

Check to see if SELINUX is disabled by running 

sestatus

Enable the time service (Chronyd)

# uncomment the following to allow the compute nodes access
# Allow NTP client access from local network.
allow 192.168.0.0/16

# add to the end of the file /etc/chrony.conf (note below for Indonesia, select correct tz)
echo "
server 1.id.pool.ntp.org
server 2.id.pool.ntp.org
server 3.id.pool.ntp.org
server 4.id.pool.ntp.org" >> /etc/chrony.conf

systemctl enable chronyd
systemctl start chronyd

Install and configure some required packages

yum -y install vim tmux wget rsyslog

Enable and start rsyslog 

systemctl enable rsyslog
systemctl start rsyslog

Setup the xCAT software repos

yum -y install yum-utils
wget --no-check-certificate -P /etc/yum.repos.d https://xcat.org/files/xcat/repos/yum/latest/xcat-core/xcat-core.repo

yum -y install centos-release-stream
wget --no-check-certificate -P /etc/yum.repos.d https://xcat.org/files/xcat/repos/yum/xcat-dep/rh8/x86_64/xcat-dep.repo

Install and provision xCAT onto management node

yum -y install xCAT 
echo ". /etc/profile.d/xcat.sh" >> ~/.bashrc
source ~/.bashrc

Configure system password for the root user on the compute nodes

chtab key=system passwd.username=root passwd.password=`openssl rand -base64 12`
  1. Note; for the password generation command use the Grave Accent (backquote)` to enclose the command rather than the apostrophe '

Set the site domain name

chdef -t site domain=dt.internal


Setup Networking for provisioning

Set the DHCP interface 

chdef -t site dhcpinterfaces="<server_name>|<server_ifname>"

Enable ssh during the installation on the compute nodes

Enables ssh to the node during installation 

chdef -t site clustersite xcatdebugmode=2
lsdef -t site clustersite

List parameters in clustersite 

lsdef -t site clustersite

Add the Centos DVD to the installer

Check the CentOS website for URL of the required version of CentOS DVD version required and then download 

wget http://mirrors.ukfast.co.uk/sites/ftp.centos.org/8.3.2011/isos/x86_64/CentOS-8.3.2011-x86_64-dvd1.iso

Once downloaded import it into the xCAT database 

copycds ./CentOS-8.3.2011-x86_64-dvd1.iso

Query available images 

lsdef -t osimage

Add a node to the installation.

mkdef -t node <node_name> groups=compute,all ip=<node_ip_address> mac=<node_mac_address> netboot=xnba arch=x86_64 bmc=<bmc_ip_address> bmcusername=<ipmi_username> bmcpassword=<ipmi_password> mgt=ipmi serialport=0 serialspeed=115200 provmethod=centos8-x86_64-install-compute

Finalise the setup

Complete network service configurations 

makehosts
makenetworks 
makedhcp -n	; Note. 'Warning: [deploy]: No dynamic range specified for 192.168.97.0. If hardware discovery is being used, a dynamic range is required.' message can be ignored as IP address is defined for the node itself.
makedns -n

Check the rcons status

systemctl status goconserver

If goconserver is not enabled or running run the appropriate commands below depending on status 

systemctl enable goconserver
systemctl start goconserver	# Not necessarily required as next command will start the server if not already running

Start goconserver if not started 

makegocons

List the console connection of the nodes 

makegocons -q

Set the nodes to pxeboot

Associate desired provisioning image for computes 

nodeset compute osimage=centos8-x86_64-install-compute
<pre>

== Precheck that the xCAT management node is ready for OS provision ==
<pre>
xcatprobe xcatmn -i <server_ifname>

Install the OS to the node and then check Installation of the node

rinstall <node_name> osimage=centos8-x86_64-install-compute # osimage is optional if it has already been defined by the provmethod of the node configuration

Once the installation is complete run the command to see the OS version installed on the node. 

xdsh '''<node_name>''' more '''/etc/*release'''

Adding additional nodes

Define the new node with the mkdef command. 

mkdef -t node <node_name> groups=compute,all ip=<node_ip_address> mac=<node_mac_address> netboot=xnba arch=x86_64 bmc=<bmc_ip_address> bmcusername=<ipmi_username> bmcpassword=<ipmi_password> mgt=ipmi serialport=0 serialspeed=115200 provmethod=centos8-x86_64-install-compute

Update the hosts file with the new node. 

makehosts <new_node_name>

Update DNS with the new node. 

makedns <new_host_name>

Install OS on the new node. 

rinstall <node_name> osimage=centos8-x86_64-install-compute

Once the installation is complete run the command to see the OS version installed on the node. 

xdsh <new_node_name> more /etc/*release

Useful commands


# check the node stats
lsdef -t node -l
lsdef -o <node_name>

# check the operating systems stats 
lsdef -t osimage -o centos8-x86_64-install-compute

# change a node mac address
makedhcp -d <nodename>
chdef -t node -o  <nodename> mac=<new-mac>
makedhcp <nodename>
lsdef -o node001

# connect to sol
rcons <nodename>
# to exit
ctrl e + c + .

# reinstall a node 
# currstate=boot
nodeset node0001 osimage=centos8-x86_64-install-compute
[root@deploy nets]# lsdef node0001 | grep currstate 
    currstate=install centos8-x86_64-compute

# using a console session - whats the root password? 
tabedit passwd

# setup RAID1 for the compute nodes
mkdir -p /install/custom/partition/
wget https://raw.githubusercontent.com/xcat2/xcat-extensions/master/partition/raid1_rh.sh -O /install/custom/partition/raid1_rh.sh

# Set the partition file as part of the provision
chdef -t osimage centos8-x86_64-install-compute partitionfile="s:/install/custom/partition/raid1_rh.sh"

# reprovision the node
rinstall node0001 osimage=centos8-x86_64-install-compute

# disk notes: check status of RAID array
cat /proc/mdstat
mdadm --detail /dev/mdX


# add a new node to an existing cluster
Retrieved from "http://wiki.define-technology.com/mediawiki-1.35.0/index.php?title=XCAT_Installation_on_Centos_8.3&oldid=32512"