VScaler: Kolla Initial setup build and deployment (Newton)

From Define Wiki
Jump to navigation Jump to search

Pre-install Notes

You'll need the following systems: (can be all-in-one, but i've split)

  1. kolla-build server (this is to build all the docker images) I've used the headnode of OpenHPC
  2. controller node(s)
  3. compute node(s)
  4. network node(s)
  5. storage node(s)

XFS Issues

Problems with XFS - Either use EXT4 or format the XFS partition with the flags below:

mkfs -t xfs -n ftype=1 /PATH/TO/DEVICE

If installing on kolla-build on a compute node, its missing loads of include files. (not a problem if using the headnode for buidling)

# NOTE: VNFS removes a lot of include files, ensure the following is commented out in:  /etc/warewulf/vnfs.conf
#hybridize += /usr/X11R6
#hybridize += /usr/lib/locale
#hybridize += /usr/lib64/locale
#hybridize += /usr/include
#hybridize += /usr/share/man
#hybridize += /usr/share/doc
#hybridize += /usr/share/locale

Missing header files on OpenHPC compute nodes

If you want to apply to a pre-deployed node and not faff around

# for some weird reason, glibc-headers is missing loads of files. Have to force an uninstall and re-install to fix on a compute node
rpm -e glibc-headers --nodeps
yum -y install glibc-headers
rpm -e glibc-devel --nodeps
yum -y install glibc-devel
rpm -e gmp-devel --nodeps
yum -y install gmp-devel
yum -y install glibc-devel
rpm -e kernel-headers --nodeps
yum -y install kernel-headers
yum -y install openssl-devel

# (WARNING BREAKS YUM) and check version is the same
rpm -e python-libs --nodeps
wget http://mirror.centos.org/centos/7/os/x86_64/Packages/python-libs-2.7.5-48.el7.x86_64.rpm
rpm -ivh ./python-libs-2.7.5-48.el7.x86_64.rpm
# end wierdness!

Setup the kolla-build system

# remove openhpc and cobbler repo if they exist!
yum -y groupinstall 'Development Tools'
yum -y update
yum -y install epel-release
yum -y install python-pip python-devel
pip install -U pip
pip -V
# At this stage im at: 
#[root@node02 yum.repos.d]# pip -V 
#pip 9.0.1 from /usr/lib/python2.7/site-packages (python 2.7)
#[root@node02 yum.repos.d]# cat /etc/redhat-release 
#CentOS Linux release 7.3.1611 (Core)

Setup Docker to act as a registry

# Install docker 
curl -sSL https://get.docker.com/ | sh
usermod -aG docker root

# Setup the sysconfig file 
tee /etc/sysconfig/docker  <<-'EOF'
DOCKER_OPTS="--insecure-registry 10.10.10.1:4000"
EOF
 mkdir -p /etc/systemd/system/docker.service.d

# setup the service file
tee /etc/systemd/system/docker.service.d/kolla.conf <<-'EOF'
[Service]
MountFlags=shared
EnvironmentFile=-/etc/sysconfig/docker
#EnvironmentFile=-/etc/sysconfig/docker-storage
#EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=
ExecStart=/usr/bin/dockerd --insecure-registry 10.10.10.1:4000
#ExecStart=/usr/bin/dockerd $DOCKER_OPTS
EOF

# restart stuff 
systemctl daemon-reload
systemctl restart docker
systemctl status docker.service
systemctl enable docker.service
systemctl stop libvirtd.service
systemctl disable libvirtd.service

# last few packages 
pip install -U docker-py
pip install ansible==2.0.0

# start up the docker registry on the deployent node 
docker run -d -p 4000:5000 --restart=always -v /var/lib/docker/volumes/registry:/var/lib/registry --name registry registry:2

Setup and Install Kolla (release 3.0.4 / Newton)

git clone https://git.openstack.org/openstack/kolla
cd kolla/
git checkout stable/newton
cd ..
# Ref problem: http://stackoverflow.com/questions/42029545/pip-is-error-typeerror-call-takes-exactly-2-arguments-1-given
# vi /usr/lib/python2.7/site-packages/packaging/requirements.py
# Changed line 59:
# #MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker")
# MARKER_EXPR = originalTextFor(MARKER_EXPR)("marker")
pip install -r kolla/requirements.txt -r kolla/test-requirements.txt
pip install kolla/
cd kolla
cp -rvf etc/kolla /etc/
yum install python-devel libffi-devel openssl-devel gcc -y
pip install -U python-openstackclient python-neutronclient
pip install tox
tox -e genconfig

Additional Notes for Ocata

git checkout stable/ocata
# then rest as above
# once kolla installed above, its only kolla-build. Need to add kolla-ansible as below
git clone https://git.openstack.org/openstack/kolla-ansible
git checkout stable/ocata
pip install kolla-ansible/

# copy conf files
cp /usr/share/kolla-ansible/etc_examples/kolla/globals.yml /etc/kolla/
cp /usr/share/kolla-ansible/etc_examples/kolla/passwords.yml /etc/kolla/
cp /root/kolla-ansible/ansible/inventory/all-in-one ~/kolla-deploy
cp /root/kolla-ansible/ansible/inventory/multinode ~/kolla-deploy

Kolla installation files - Multinode

Create (copy) the multinode inventory file:

#setup the installation / build file
mkdir ~/kolla-deploy
cd ~/kolla-deploy
cp /usr/share/kolla/ansible/inventory/multinode ~/kolla-deploy

Edit the multinode inventory file. It should have something like this:

[control]
# These hostname must be resolvable from your deployment host
controller01
controller02
controller03

# The above can also be specified as follows:
#controller[01:03]     ansible_user=kolla

# The network nodes are where your l3-agent and loadbalancers will run
# This can be the same as a host in the control group
[network]
controller01
controller02
controller03

[compute]
compute01
compute02
compute03
compute04
gpu01 api_interface=ens5

[monitoring]
controller01
controller02
controller03

# When compute nodes and control nodes use different interfaces,
# you can specify "api_interface" and other interfaces like below:
#compute01 neutron_external_interface=eth0 api_interface=em1 storage_interface=em1 tunnel_interface=em1

[storage]
storage01
storage02
storage03
storage04
.
.
.

NB - GPU01 in our setup above has a different network interface name to the rest. The common interface name is defined in globals.yml and the exception is declared above in the multinode file.

Kolla installation files - /etc/kolla/passwords

kolla-genpwd

Kolla installation files - /etc/kolla/globals.yml

Make sure you define entries in /etc/kolla/globals.yml as below.

  • This was for my 1GbE setup:
[root@head01 ~]# egrep -v '(^#|^$)' /etc/kolla/globals.yml.1gbe 
---
kolla_internal_vip_address: "10.10.10.254"
docker_registry: "10.10.10.1:4000"
network_interface: "eth2"
neutron_external_interface: "eno2"
ceph_enable_cache: "yes"
ceph_cache_mode: "writeback"
ceph_pool_type: "erasure"
designate_backend: "bind9"
designate_ns_record: "sample.openstack.org"
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:
  • This was Dave's original:
[root@gpu01 kolla]# egrep -v '(^#|^$)' /etc/kolla/globals.yml
---
kolla_internal_vip_address: "10.10.10.254"
docker_registry: "10.10.10.1:4000"
network_interface: "eth2"
neutron_external_interface: "eno1"
  • And this was for my 10GbE setup:
[root@head01 ~]# egrep -v '(^#|^$)' /etc/kolla/globals.yml
---
kolla_internal_vip_address: "192.168.50.254"
docker_registry: "10.10.10.1:4000"
network_interface: "enp2s0"
neutron_external_interface: "eno2"
ceph_enable_cache: "yes"
ceph_cache_mode: "writeback"
ceph_pool_type: "erasure"
designate_backend: "bind9"
designate_ns_record: "sample.openstack.org"
tempest_image_id:
tempest_flavor_ref_id:
tempest_public_network_id:
tempest_floating_network_name:

Build some docker images

kolla-build --base centos --type binary --registry 10.10.10.1:4000 --push
# some will fail because binary candidates not found - you can build from source using (for e.g. murano / mistral didnt build)
# kolla-build --base centos --type source --registry 10.10.10.1:4000 --push

check the docker image registry for all your images

docker images -a 
docker ps -a

Update Ansible and disable the standard libvirtd

ansible version 2.0.0.0 resulted in kolla-ansible bootstrap and precheck error, so

pip install -U ansible

Make sure libvirtd is stopped and disabled on compute nodes

pdsh -w compute0[1-4],gpu01 'systemctl stop libvirtd && systemctl disable libvirtd'

Deploy and Test

kolla-ansible bootstrap-servers -i all-in-one 
kolla-ansible prechecks -i ./multinode
kolla-ansible pull -i ./multinode # make sure you can pull all the images 
kolla-ansible -vvv deploy -i ./multinode
kolla-ansible post-deploy # this creates the admin-openrc.sh, in /etc/kolla, credentials
# sometime you neeed add or remove a computer node or controller node, you should modify /usr/share/kolla/ansible/inventory/multinode, then execute:
# kolla-ansible upgrade -i ./multinode
pip install python-openstackclient
source /etc/kolla/admin-openrc.sh
/usr/share/kolla/init-runonce

Configure CEPH

The disk setup on our storage nodes:

[root@head01 ~]# ssh storage01 lsscsi
[0:0:0:0]    disk    ATA      TOSHIBA MG04ACA6 FS2B  /dev/sda 
[0:0:1:0]    disk    ATA      TOSHIBA MG04ACA6 FS2B  /dev/sdb 
[0:0:2:0]    disk    ATA      TOSHIBA MG04ACA6 FS2B  /dev/sdc 
[0:0:3:0]    disk    ATA      TOSHIBA MG04ACA6 FS2B  /dev/sdd 
[0:0:4:0]    disk    ATA      SDLFGC7R-960G-1H MC99  /dev/sde 
[0:0:5:0]    disk    ATA      TOSHIBA MG04ACA6 FS2B  /dev/sdf 
[0:0:7:0]    disk    ATA      TOSHIBA MG04ACA6 FS2B  /dev/sdg

And partition layout (the O/S is installed on /dev/sda):

[root@head01 ~]# ssh storage01 df -h
Filesystem                Size  Used Avail Use% Mounted on
/dev/sda3                 2.0T  3.9G  1.9T   1% /
devtmpfs                   16G     0   16G   0% /dev
tmpfs                      16G     0   16G   0% /dev/shm
tmpfs                      16G   17M   16G   1% /run
tmpfs                      16G     0   16G   0% /sys/fs/cgroup
/dev/sda1                 463M  113M  322M  26% /boot
10.10.10.1:/home          909G   34G  830G   4% /home
10.10.10.1:/opt/ohpc/pub  909G   34G  830G   4% /opt/ohpc/pub
tmpfs                     3.2G     0  3.2G   0% /run/user/0

Label each disk to be used as an OSD

In our setup, we're going to use /dev/sd[b-d,f-g] as OSDs.

[root@head01 ~]# pdsh -w storage0[1-4] 'id=0; for i in "sdb" "sdc" "sdd" "sdf" "sdg"; do parted /dev/$i -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_BOOTSTRAP_FO${id} 1 -1;((id++)) ; done'

Check that worked:

[root@head01 ~]# pdsh -w storage0[1-4] 'id=0; for i in "sdb" "sdc" "sdd" "sdf" "sdg"; do parted /dev/$i print | grep FO;((id++)) ; done'
storage01:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO0
storage02:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO0
storage04:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO0
storage03:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO0
storage01:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO1
storage03:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO1
storage02:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO1
storage04:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO1
storage01:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO2
storage03:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO2
storage02:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO2
storage01:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO3
storage04:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO2
storage04:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO3
storage03:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO3
storage02:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO3
storage01:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO4
storage03:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO4
storage04:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO4
storage02:  1      1049kB  6001GB  6001GB               KOLLA_CEPH_OSD_BOOTSTRAP_FO4

Label each SSD to be used as a cache drive

In our setup, we're going to use /dev/sde as a cache drive.

[root@head01 ~]# pdsh -w storage0[1-4] 'parted /dev/sde -s -- mklabel gpt mkpart KOLLA_CEPH_OSD_CACHE_BOOTSTRAP 1 -1'

Check that worked:

[root@head01 ~]# pdsh -w storage0[1-4] 'parted /dev/sde print | grep -i cache'
storage01:  1      1049kB  960GB  960GB               KOLLA_CEPH_OSD_CACHE_BOOTSTRAP
storage02:  1      1049kB  960GB  960GB               KOLLA_CEPH_OSD_CACHE_BOOTSTRAP
storage04:  1      1049kB  960GB  960GB               KOLLA_CEPH_OSD_CACHE_BOOTSTRAP
storage03:  1      1049kB  960GB  960GB               KOLLA_CEPH_OSD_CACHE_BOOTSTRAP

Enable CEPH

Enable CEPH in /etc/kolla/globals.yml:

enable_ceph: "yes"

Enable RadosGW in /etc/kolla/globals.yml also (this is optional):

enable_ceph_rgw: "yes"

Enable cache tier in /etc/kolla/globals.yml:

enable_ceph: "yes"
ceph_enable_cache: "yes"
# Valid options are [ forward, none, writeback ]
ceph_cache_mode: "writeback"

To enable erasure coded pools add the following options to your /etc/kolla/globals.yml configuration file (this is optional (the default is 'replicated') but a cache tier is required for erasure coding):

# A requirement for using the erasure-coded pools is you must setup a cache tier
# Valid options are [ erasure, replicated ]
ceph_pool_type: "erasure"
# Optionally, you can change the profile
#ceph_erasure_profile: "k=4 m=2 ruleset-failure-domain=host"

Sample Docker Images Output

[root@mitaka-head ~]# docker images 
REPOSITORY                                                       TAG                 IMAGE ID            CREATED             SIZE
10.10.10.1:4000/kolla/centos-binary-magnum-api                   3.0.4               d78bd203cff2        3 days ago          736 MB
10.10.10.1:4000/kolla/centos-binary-magnum-conductor             3.0.4               b8f647207273        3 days ago          778 MB
10.10.10.1:4000/kolla/centos-binary-magnum-base                  3.0.4               06da81d5dcc0        3 days ago          713 MB
10.10.10.1:4000/kolla/centos-binary-openstack-base               3.0.4               d5f53a04be6f        3 days ago          563 MB
10.10.10.1:4000/kolla/centos-binary-cinder-api                   3.0.4               70f3a5e554f5        4 days ago          908 MB
10.10.10.1:4000/kolla/centos-binary-cinder-backup                3.0.4               aa042081cd42        4 days ago          896 MB
10.10.10.1:4000/kolla/centos-binary-cinder-volume                3.0.4               5ea20d978a44        4 days ago          869 MB
10.10.10.1:4000/kolla/centos-binary-cinder-rpcbind               3.0.4               2dcf7f6a52f0        4 days ago          896 MB
10.10.10.1:4000/kolla/centos-binary-nova-compute                 3.0.4               f36118d8d566        4 days ago          1.18 GB
10.10.10.1:4000/kolla/centos-binary-gnocchi-metricd              3.0.4               6bade29616fd        4 days ago          900 MB
10.10.10.1:4000/kolla/centos-binary-gnocchi-api                  3.0.4               16dbd9030f28        4 days ago          900 MB
10.10.10.1:4000/kolla/centos-binary-gnocchi-statsd               3.0.4               66a01dfe0086        4 days ago          900 MB
10.10.10.1:4000/kolla/centos-binary-cinder-scheduler             3.0.4               dca0e01ea4b2        4 days ago          867 MB
10.10.10.1:4000/kolla/centos-binary-trove-api                    3.0.4               cd2ea7c5daab        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-nova-libvirt                 3.0.4               2624b17fd29b        4 days ago          1.17 GB
10.10.10.1:4000/kolla/centos-binary-trove-conductor              3.0.4               c373b659c4c8        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-trove-guestagent             3.0.4               29de10abdab6        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-sahara-api                   3.0.4               610d007d006d        4 days ago          624 MB
10.10.10.1:4000/kolla/centos-binary-trove-taskmanager            3.0.4               313e89f059af        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-sahara-engine                3.0.4               e2ce45f721e0        4 days ago          624 MB
10.10.10.1:4000/kolla/centos-binary-nova-compute-ironic          3.0.4               110f5267fa67        4 days ago          1.09 GB
10.10.10.1:4000/kolla/centos-binary-designate-pool-manager       3.0.4               9443ac730c1a        4 days ago          650 MB
10.10.10.1:4000/kolla/centos-binary-designate-central            3.0.4               0fc8feaddace        4 days ago          643 MB
10.10.10.1:4000/kolla/centos-binary-designate-backend-bind9      3.0.4               3fa8aad9af41        4 days ago          650 MB
10.10.10.1:4000/kolla/centos-binary-designate-sink               3.0.4               51e17ddc0aa3        4 days ago          643 MB
10.10.10.1:4000/kolla/centos-binary-designate-api                3.0.4               af28ee2b73d7        4 days ago          643 MB
10.10.10.1:4000/kolla/centos-binary-ironic-conductor             3.0.4               ff90f3d51d4e        4 days ago          734 MB
10.10.10.1:4000/kolla/centos-binary-ironic-inspector             3.0.4               1415866fc44e        4 days ago          628 MB
10.10.10.1:4000/kolla/centos-binary-designate-mdns               3.0.4               9f412c0057fb        4 days ago          643 MB
10.10.10.1:4000/kolla/centos-binary-ironic-api                   3.0.4               5ea4be02bcf8        4 days ago          623 MB
10.10.10.1:4000/kolla/centos-binary-ironic-pxe                   3.0.4               d4485a2ec362        4 days ago          626 MB
10.10.10.1:4000/kolla/centos-binary-swift-proxy-server           3.0.4               e1572d4c0881        4 days ago          611 MB
10.10.10.1:4000/kolla/centos-binary-swift-container              3.0.4               9fc6f2c27987        4 days ago          611 MB
10.10.10.1:4000/kolla/centos-binary-swift-account                3.0.4               5ab93fed4cb1        4 days ago          611 MB
10.10.10.1:4000/kolla/centos-binary-swift-object-expirer         3.0.4               0dc90490613e        4 days ago          611 MB
10.10.10.1:4000/kolla/centos-binary-swift-rsyncd                 3.0.4               a503a9d894d5        4 days ago          592 MB
10.10.10.1:4000/kolla/centos-binary-swift-object                 3.0.4               9d36f1d4be69        4 days ago          611 MB
10.10.10.1:4000/kolla/centos-binary-neutron-openvswitch-agent    3.0.4               d9eea5cbaf66        4 days ago          669 MB
10.10.10.1:4000/kolla/centos-binary-neutron-metadata-agent       3.0.4               cf108ab59425        4 days ago          646 MB
10.10.10.1:4000/kolla/centos-binary-neutron-lbaas-agent          3.0.4               53e4646f80f6        4 days ago          671 MB
10.10.10.1:4000/kolla/centos-binary-neutron-dhcp-agent           3.0.4               478151798644        4 days ago          646 MB
10.10.10.1:4000/kolla/centos-binary-neutron-l3-agent             3.0.4               478151798644        4 days ago          646 MB
10.10.10.1:4000/kolla/centos-binary-neutron-linuxbridge-agent    3.0.4               222faa99b714        4 days ago          669 MB
10.10.10.1:4000/kolla/centos-binary-neutron-server               3.0.4               8f1525d66677        4 days ago          669 MB
10.10.10.1:4000/kolla/centos-binary-neutron-metering-agent       3.0.4               2cd99816449a        4 days ago          669 MB
10.10.10.1:4000/kolla/centos-binary-neutron-vpnaas-agent         3.0.4               f0544ff6f83f        4 days ago          677 MB
10.10.10.1:4000/kolla/centos-binary-nova-spicehtml5proxy         3.0.4               3a13a9952957        4 days ago          706 MB
10.10.10.1:4000/kolla/centos-binary-nova-novncproxy              3.0.4               aaefaa06bc23        4 days ago          666 MB
10.10.10.1:4000/kolla/centos-binary-nova-conductor               3.0.4               3ab8b021c390        4 days ago          665 MB
<none>                                                           <none>              41e68f24c58c        4 days ago          646 MB
10.10.10.1:4000/kolla/centos-binary-nova-ssh                     3.0.4               fad46d27b2f2        4 days ago          666 MB
10.10.10.1:4000/kolla/centos-binary-nova-scheduler               3.0.4               6125313f86cb        4 days ago          665 MB
10.10.10.1:4000/kolla/centos-binary-nova-consoleauth             3.0.4               00e3f0071755        4 days ago          665 MB
10.10.10.1:4000/kolla/centos-binary-nova-api                     3.0.4               f1fe496bcfe9        4 days ago          665 MB
10.10.10.1:4000/kolla/centos-binary-nova-network                 3.0.4               613ae54d2c96        4 days ago          643 MB
10.10.10.1:4000/kolla/centos-binary-nova-serialproxy             3.0.4               1a6b57cc8494        4 days ago          665 MB
10.10.10.1:4000/kolla/centos-binary-heat-api                     3.0.4               0f8b6f127351        4 days ago          618 MB
10.10.10.1:4000/kolla/centos-binary-heat-engine                  3.0.4               de851441fc42        4 days ago          618 MB
10.10.10.1:4000/kolla/centos-binary-heat-api-cfn                 3.0.4               cc82d8ac423a        4 days ago          618 MB
10.10.10.1:4000/kolla/centos-binary-manila-share                 3.0.4               8964a2d55a60        4 days ago          659 MB
10.10.10.1:4000/kolla/centos-binary-magnum-api                   <none>              e8df6d116e47        4 days ago          736 MB
10.10.10.1:4000/kolla/centos-binary-magnum-conductor             <none>              7fa1bac8fc26        4 days ago          778 MB
10.10.10.1:4000/kolla/centos-binary-murano-api                   3.0.4               af3c3b0e65e7        4 days ago          609 MB
10.10.10.1:4000/kolla/centos-binary-ceilometer-api               3.0.4               3650b46a3bd4        4 days ago          658 MB
10.10.10.1:4000/kolla/centos-binary-ceilometer-central           3.0.4               8049b28120c6        4 days ago          658 MB
10.10.10.1:4000/kolla/centos-binary-manila-scheduler             3.0.4               39394a79139b        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-manila-api                   3.0.4               ba7857422288        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-ceilometer-compute           3.0.4               a69956792ab9        4 days ago          658 MB
10.10.10.1:4000/kolla/centos-binary-murano-engine                3.0.4               8cd8126764fa        4 days ago          609 MB
10.10.10.1:4000/kolla/centos-binary-ceilometer-collector         3.0.4               cd82da011309        4 days ago          619 MB
10.10.10.1:4000/kolla/centos-binary-ceilometer-notification      3.0.4               c0e64f186be1        4 days ago          618 MB
10.10.10.1:4000/kolla/centos-binary-cinder-base                  3.0.4               eee63fa966a2        4 days ago          867 MB
10.10.10.1:4000/kolla/centos-binary-glance-api                   3.0.4               dd54b3d0a8dd        4 days ago          767 MB
10.10.10.1:4000/kolla/centos-binary-aodh-api                     3.0.4               ac14af310016        4 days ago          653 MB
10.10.10.1:4000/kolla/centos-binary-aodh-notifier                3.0.4               9971ea02b3ce        4 days ago          653 MB
10.10.10.1:4000/kolla/centos-binary-aodh-listener                3.0.4               86b95ea7357e        4 days ago          653 MB
10.10.10.1:4000/kolla/centos-binary-glance-registry              3.0.4               d5deb86446be        4 days ago          727 MB
10.10.10.1:4000/kolla/centos-binary-gnocchi-base                 3.0.4               d21049a6ee98        4 days ago          874 MB
10.10.10.1:4000/kolla/centos-binary-aodh-evaluator               3.0.4               d69194a554ea        4 days ago          653 MB
10.10.10.1:4000/kolla/centos-binary-aodh-expirer                 3.0.4               bcc23ba2efb5        4 days ago          653 MB
10.10.10.1:4000/kolla/centos-binary-mistral-api                  3.0.4               a0e6640e2a7a        4 days ago          721 MB
10.10.10.1:4000/kolla/centos-binary-mistral-executor             3.0.4               24d3b9511f3b        4 days ago          721 MB
10.10.10.1:4000/kolla/centos-binary-mistral-engine               3.0.4               7492b71c6427        4 days ago          721 MB
10.10.10.1:4000/kolla/centos-binary-keystone-fernet              3.0.4               74ab0570f7d3        4 days ago          655 MB
10.10.10.1:4000/kolla/centos-binary-keystone-ssh                 3.0.4               75dc526d7831        4 days ago          655 MB
10.10.10.1:4000/kolla/centos-binary-barbican-api                 3.0.4               df4ac8990165        4 days ago          617 MB
10.10.10.1:4000/kolla/centos-binary-keystone                     3.0.4               2432e09e913e        4 days ago          633 MB
10.10.10.1:4000/kolla/centos-binary-barbican-keystone-listener   3.0.4               2eacdcfa1542        4 days ago          615 MB
10.10.10.1:4000/kolla/centos-binary-barbican-worker              3.0.4               026000cbe4b9        4 days ago          615 MB
10.10.10.1:4000/kolla/centos-binary-trove-base                   3.0.4               5483308bac67        4 days ago          593 MB
10.10.10.1:4000/kolla/centos-binary-sahara-base                  3.0.4               34ec94aa1835        4 days ago          603 MB
10.10.10.1:4000/kolla/centos-binary-designate-base               3.0.4               ed346abbe9b9        4 days ago          621 MB
10.10.10.1:4000/kolla/centos-binary-ironic-base                  3.0.4               181c88398e5b        4 days ago          602 MB
10.10.10.1:4000/kolla/centos-binary-swift-base                   3.0.4               1adc3171914a        4 days ago          590 MB
10.10.10.1:4000/kolla/centos-binary-neutron-base                 3.0.4               1b289b46f068        4 days ago          646 MB
10.10.10.1:4000/kolla/centos-binary-nova-base                    3.0.4               2de65d31d0d5        4 days ago          643 MB
10.10.10.1:4000/kolla/centos-binary-zaqar                        3.0.4               056bdfbf8c45        4 days ago          624 MB
10.10.10.1:4000/kolla/centos-binary-heat-base                    3.0.4               088b1de82840        4 days ago          597 MB
10.10.10.1:4000/kolla/centos-binary-rally                        3.0.4               378e6d729a1c        4 days ago          617 MB
10.10.10.1:4000/kolla/centos-binary-collectd                     3.0.4               96c68198b1c1        4 days ago          586 MB
10.10.10.1:4000/kolla/centos-binary-horizon                      3.0.4               e66ecb72a90c        4 days ago          798 MB
10.10.10.1:4000/kolla/centos-binary-manila-base                  3.0.4               744aab43c0a6        4 days ago          614 MB
10.10.10.1:4000/kolla/centos-binary-magnum-base                  <none>              fe915ea749d0        4 days ago          713 MB
10.10.10.1:4000/kolla/centos-binary-murano-base                  3.0.4               79e425868dc9        4 days ago          588 MB
10.10.10.1:4000/kolla/centos-binary-ceilometer-base              3.0.4               6aa09e03f06e        4 days ago          597 MB
10.10.10.1:4000/kolla/centos-binary-tempest                      3.0.4               854131b7291c        4 days ago          642 MB
10.10.10.1:4000/kolla/centos-binary-glance-base                  3.0.4               a86429091688        4 days ago          727 MB
10.10.10.1:4000/kolla/centos-binary-aodh-base                    3.0.4               6d8952a93c01        4 days ago          631 MB
10.10.10.1:4000/kolla/centos-binary-mistral-base                 3.0.4               7312fbbf0859        4 days ago          698 MB
10.10.10.1:4000/kolla/centos-binary-keystone-base                3.0.4               87b5b322b9c7        4 days ago          633 MB
10.10.10.1:4000/kolla/centos-binary-dind                         3.0.4               f44486475b3e        4 days ago          563 MB
10.10.10.1:4000/kolla/centos-binary-barbican-base                3.0.4               3e5e7a8f6fba        4 days ago          594 MB
10.10.10.1:4000/kolla/centos-binary-kolla-toolbox                3.0.4               2cd463f3cbce        4 days ago          712 MB
<none>                                                           <none>              cc72267144fa        4 days ago          563 MB
10.10.10.1:4000/kolla/centos-binary-openstack-base               <none>              207b9fd4155c        4 days ago          563 MB
10.10.10.1:4000/kolla/centos-binary-ceph-mon                     3.0.4               0793b4e7ab13        4 days ago          680 MB
10.10.10.1:4000/kolla/centos-binary-mariadb                      3.0.4               b998b4aabcee        4 days ago          789 MB
10.10.10.1:4000/kolla/centos-binary-ceph-osd                     3.0.4               3d97cb4483cb        4 days ago          680 MB
10.10.10.1:4000/kolla/centos-binary-grafana                      3.0.4               264841c74653        4 days ago          536 MB
10.10.10.1:4000/kolla/centos-binary-ceph-rgw                     3.0.4               17fffb4aa582        4 days ago          680 MB
10.10.10.1:4000/kolla/centos-binary-ceph-base                    3.0.4               5b1dbb3bb9ff        4 days ago          680 MB
10.10.10.1:4000/kolla/centos-binary-openvswitch-db-server        3.0.4               9b46efbc7f66        4 days ago          410 MB
10.10.10.1:4000/kolla/centos-binary-iscsid                       3.0.4               b34845092c83        4 days ago          403 MB
10.10.10.1:4000/kolla/centos-binary-openvswitch-vswitchd         3.0.4               782e4c3db61f        4 days ago          410 MB
10.10.10.1:4000/kolla/centos-binary-memcached                    3.0.4               061fe09dfd0a        4 days ago          400 MB
10.10.10.1:4000/kolla/centos-binary-kibana                       3.0.4               78c3c2f08ae1        4 days ago          534 MB
10.10.10.1:4000/kolla/centos-binary-heka                         3.0.4               5be809a8f701        4 days ago          453 MB
10.10.10.1:4000/kolla/centos-binary-elasticsearch                3.0.4               20582ad45b37        4 days ago          607 MB
10.10.10.1:4000/kolla/centos-binary-dnsmasq                      3.0.4               5ede222f22fe        4 days ago          399 MB
10.10.10.1:4000/kolla/centos-binary-keepalived                   3.0.4               61ef2dd501bb        4 days ago          405 MB
10.10.10.1:4000/kolla/centos-binary-telegraf                     3.0.4               fb74d4f362fe        4 days ago          432 MB
10.10.10.1:4000/kolla/centos-binary-rabbitmq                     3.0.4               4949ffe59699        4 days ago          440 MB
10.10.10.1:4000/kolla/centos-binary-etcd                         3.0.4               26799eb72e7a        4 days ago          428 MB
10.10.10.1:4000/kolla/centos-binary-openvswitch-base             3.0.4               23f4c55b8dd6        4 days ago          410 MB
10.10.10.1:4000/kolla/centos-binary-haproxy                      3.0.4               8c4b31f52f01        4 days ago          420 MB
10.10.10.1:4000/kolla/centos-binary-influxdb                     3.0.4               0bfccba88174        4 days ago          454 MB
10.10.10.1:4000/kolla/centos-binary-mongodb                      3.0.4               a55285903cbf        4 days ago          571 MB
10.10.10.1:4000/kolla/centos-binary-cron                         3.0.4               1e7ec4979bd3        4 days ago          399 MB
10.10.10.1:4000/kolla/centos-binary-tgtd                         3.0.4               8110cd117d81        4 days ago          381 MB
10.10.10.1:4000/kolla/centos-binary-multipathd                   3.0.4               b50eb31c130c        4 days ago          409 MB
10.10.10.1:4000/kolla/centos-binary-base                         3.0.4               c33393731ca5        4 days ago          379 MB
centos                                                           7                   98d35105a391        2 weeks ago         192 MB
registry                                                         2                   047218491f8c        4 weeks ago         33.2 MB

Port Forwards

firewall-cmd --zone=external --add-forward-port=port=443:proto=tcp:toport=443:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=80:proto=tcp:toport=80:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=6080:proto=tcp:toport=6080:toaddr=10.10.10.254 --permanent

firewall-cmd --zone=external --add-forward-port=port=5000:proto=tcp:toport=5000:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=35357:proto=tcp:toport=35357:toaddr=10.10.10.254 --permanent

firewall-cmd --zone=external --add-forward-port=port=8776:proto=tcp:toport=8776:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=8774:proto=tcp:toport=8774:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=8386:proto=tcp:toport=8386:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=9292:proto=tcp:toport=9292:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=9696:proto=tcp:toport=9696:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=6000:proto=tcp:toport=6000:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=6001:proto=tcp:toport=6001:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=6002:proto=tcp:toport=6002:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=8004:proto=tcp:toport=8004:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=8777:proto=tcp:toport=8777:toaddr=10.10.10.254 --permanent
firewall-cmd --zone=external --add-forward-port=port=8000:proto=tcp:toport=8000:toaddr=10.10.10.254 --permanent

firewall-cmd --reload