Adding Storpool Storage to OpenStack - Zed release
Jump to navigation
Jump to search
Get the Storpool containers
# storpool images downloaded define@kolla-deploy:~$ docker images | grep storpool cts.storpool.com/kolla/ubuntu-binary-nova-compute zed-ubuntu-jammy.20230309.2 f4fae03411f8 2 weeks ago 1.48GB cts.storpool.com/kolla/ubuntu-binary-cinder-volume zed-ubuntu-jammy.20230309.2 959e8af980ca 2 weeks ago 1.36GB
Check the naming used on Zed
# whats the current nova zed naming define@kolla-deploy:~$ docker images | grep nova | grep zed cts.storpool.com/kolla/ubuntu-binary-nova-compute zed-ubuntu-jammy.20230309.2 f4fae03411f8 2 weeks ago 1.48GB registry.api.internal:5000/kolla/nova-compute zed-ubuntu-jammy fc13cf204b3c 3 weeks ago 1.47GB registry.maas:5000/kolla/nova-compute zed-ubuntu-jammy fc13cf204b3c 3 weeks ago 1.47GB registry.api.internal:5000/kolla/nova-novncproxy zed-ubuntu-jammy 9d46649e580b 3 weeks ago 1.22GB registry.maas:5000/kolla/nova-novncproxy zed-ubuntu-jammy 9d46649e580b 3 weeks ago 1.22GB registry.api.internal:5000/kolla/nova-spicehtml5proxy zed-ubuntu-jammy 179707acb6ab 3 weeks ago 1.15GB registry.maas:5000/kolla/nova-spicehtml5proxy zed-ubuntu-jammy 179707acb6ab 3 weeks ago 1.15GB registry.maas:5000/kolla/nova-libvirt zed-ubuntu-jammy accca5769ec4 3 weeks ago 987MB registry.maas:5000/kolla/nova-serialproxy zed-ubuntu-jammy aebf88171aeb 3 weeks ago 1.11GB registry.maas:5000/kolla/nova-conductor zed-ubuntu-jammy 1274b18ec7f9 3 weeks ago 1.11GB registry.maas:5000/kolla/nova-compute-ironic zed-ubuntu-jammy b23d537996d3 3 weeks ago 1.12GB registry.maas:5000/kolla/nova-ssh zed-ubuntu-jammy 3b74d7d5b22f 3 weeks ago 1.11GB registry.maas:5000/kolla/nova-scheduler zed-ubuntu-jammy 80b82e6218a7 3 weeks ago 1.11GB registry.maas:5000/kolla/nova-api zed-ubuntu-jammy 1509aaad9a2f 3 weeks ago 1.11GB registry.maas:5000/kolla/nova-base zed-ubuntu-jammy 24460ee3b28a 3 weeks ago 1.11GB registry.maas:5000/nova-compute zed-ubuntu-jammy ed81b01df5db 7 weeks ago 1.47GB define@kolla-deploy:~$
Tag the images
# lets tag with a -storpool at end define@kolla-deploy:~$ docker tag cts.storpool.com/kolla/ubuntu-binary-nova-compute:zed-ubuntu-jammy.20230309.2 registry.maas:5000/kolla/nova-compute:zed-ubuntu-jammy-storpool
# lets check cinder naming define@kolla-deploy:~$ docker images | grep cinder | grep zed cts.storpool.com/kolla/ubuntu-binary-cinder-volume zed-ubuntu-jammy.20230309.2 959e8af980ca 2 weeks ago 1.36GB registry.maas:5000/kolla/cinder-volume zed-ubuntu-jammy 62fcdfbcfbc8 3 weeks ago 1.36GB registry.maas:5000/kolla/cinder-backup zed-ubuntu-jammy c5a4ea38f3d8 3 weeks ago 1.35GB registry.maas:5000/kolla/cinder-scheduler zed-ubuntu-jammy 667c90a72c25 3 weeks ago 1.32GB registry.maas:5000/kolla/cinder-api zed-ubuntu-jammy cf603f7615ec 3 weeks ago 1.32GB registry.maas:5000/kolla/cinder-base zed-ubuntu-jammy 89d7a6973593 3 weeks ago 1.32GB
# tag cinder; define@kolla-deploy:~$ docker tag cts.storpool.com/kolla/ubuntu-binary-cinder-volume:zed-ubuntu-jammy.20230309.2 registry.maas:5000/kolla/cinder-volume:zed-ubuntu-jammy-storpool define@kolla-deploy:~$ docker images | grep storpool cts.storpool.com/kolla/ubuntu-binary-nova-compute zed-ubuntu-jammy.20230309.2 f4fae03411f8 2 weeks ago 1.48GB registry.maas:5000/kolla/nova-compute zed-ubuntu-jammy-storpool f4fae03411f8 2 weeks ago 1.48GB cts.storpool.com/kolla/ubuntu-binary-cinder-volume zed-ubuntu-jammy.20230309.2 959e8af980ca 2 weeks ago 1.36GB registry.maas:5000/kolla/cinder-volume zed-ubuntu-jammy-storpool 959e8af980ca 2 weeks ago 1.36GB
Push the tagged containers
# push containers 1996 docker push registry.maas:5000/kolla/cinder-volume:zed-ubuntu-jammy-storpool 1998 docker push registry.maas:5000/kolla/nova-compute:zed-ubuntu-jammy-storpool
Storpool configuration files setup
# no storpool files exists so lets touch define@kolla-deploy:~$ ssh ubuntu@inf-233 sudo touch /etc/storpool.conf define@kolla-deploy:~$ ssh ubuntu@inf-234 sudo touch /etc/storpool.conf define@kolla-deploy:~$ ssh ubuntu@inf-235 sudo touch /etc/storpool.conf
# config file globals.yml enable_cinder: "yes" enable_cinder_backup: "yes" # storpool dp enable_cinder_backend_iscsi: "yes" enable_cinder_backend_lvm: "no" skip_cinder_backend_check: True cinder_volume_image: "registry.maas:5000/kolla/cinder-volume" cinder_volume_tag: "zed-ubuntu-jammy-storpool" cinder_volume_extra_volumes: - "/etc/storpool.conf:/etc/storpool.conf:ro" # end storpool
Get the iscsid stuff setup
# get the iqns define@kolla-deploy:~$ ssh ubuntu@inf-233 sudo systemctl enable --now iscsid Synchronizing state of iscsid.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable iscsid Failed to enable unit: Unit file /etc/systemd/system/iscsid.service is masked. define@kolla-deploy:~$ ssh ubuntu@inf-234 sudo systemctl enable --now iscsid Synchronizing state of iscsid.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable iscsid Failed to enable unit: Unit file /etc/systemd/system/iscsid.service is masked. define@kolla-deploy:~$ ssh ubuntu@inf-235 sudo systemctl enable --now iscsid Synchronizing state of iscsid.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable iscsid Failed to enable unit: Unit file /etc/systemd/system/iscsid.service is masked. # missing iscsi systemctl unmask iscsid apt install -y open-iscsi systemctl restart iscsid systemctl status iscsid cat /etc/iscsi/initiatorname.iscsi
Setup the configuration cinder.conf
# cinder.conf [DEFAULT] enabled_backends=ceph-replicated, storpool #default_volume_type = ceph-fast [storpool] volume_driver = cinder.volume.drivers.storpool.StorPoolDriver volume_backend_name = storpool iscsi_cinder_volume = True iscsi_portal_group = "pg" iscsi_export_to = " iqn.2004-10.com.ubuntu:01:7560187358d1 iqn.2004-10.com.ubuntu:01:d15fac349d89 iqn.2004-10.com.ubuntu:01:65be63696e " image_upload_use_cinder_backend = True image_upload_use_internal_tenant = True
# add control to the iscsi investory (kolla-ansible-zed) define@kolla-deploy:~/kolla/config$ grep iscsi inventory/groups -A 5 [iscsid:children] compute storage ironic control
Deploy
# ok added the iqns – lets deploy kolla-ansible --configdir=$(pwd) -i inventory -t cinder deploy
Check the container status
# check the rollout (kolla-ansible-zed) define@kolla-deploy:~/kolla/config$ ssh ubuntu@inf-233 sudo docker ps '| grep cinder' 71ee80e283cc registry.maas:5000/kolla/cinder-volume:zed-ubuntu-jammy-storpool "dumb-init --single-…" 2 minutes ago Up 2 minutes (healthy) cinder_volume 0a29f3a47726 registry.maas:5000/kolla/cinder-backup:zed-ubuntu-jammy "dumb-init --single-…" 8 minutes ago Up 2 minutes (healthy) cinder_backup 635873d31c5d registry.maas:5000/kolla/cinder-scheduler:zed-ubuntu-jammy "dumb-init --single-…" 2 weeks ago Up 2 minutes (healthy) cinder_scheduler 465f48acac9e registry.maas:5000/kolla/cinder-api:zed-ubuntu-jammy "dumb-init --single-…" 2 weeks ago Up 2 minutes (healthy) cinder_api (kolla-ansible-zed) define@kolla-deploy:~/kolla/config$ ssh ubuntu@inf-234 sudo docker ps '| grep cinder' 82d85ac4bcd0 registry.maas:5000/kolla/cinder-volume:zed-ubuntu-jammy-storpool "dumb-init --single-…" 2 minutes ago Up 2 minutes (healthy) cinder_volume 5eeaeda3c600 registry.maas:5000/kolla/cinder-backup:zed-ubuntu-jammy "dumb-init --single-…" 8 minutes ago Up 2 minutes (healthy) cinder_backup 2415fef21f0a registry.maas:5000/kolla/cinder-scheduler:zed-ubuntu-jammy "dumb-init --single-…" 2 weeks ago Up 2 minutes (healthy) cinder_scheduler c07f624be1c4 registry.maas:5000/kolla/cinder-api:zed-ubuntu-jammy "dumb-init --single-…" 2 weeks ago Up 2 minutes (healthy) cinder_api (kolla-ansible-zed) define@kolla-deploy:~/kolla/config$ ssh ubuntu@inf-235 sudo docker ps '| grep cinder' 9578d8a23f34 registry.maas:5000/kolla/cinder-volume:zed-ubuntu-jammy-storpool "dumb-init --single-…" 2 minutes ago Up 2 minutes (healthy) cinder_volume 2d877f6cff8e registry.maas:5000/kolla/cinder-backup:zed-ubuntu-jammy "dumb-init --single-…" 8 minutes ago Up 2 minutes (healthy) cinder_backup 8bd93484fa5a registry.maas:5000/kolla/cinder-scheduler:zed-ubuntu-jammy "dumb-init --single-…" 2 weeks ago Up 2 minutes (healthy) cinder_scheduler b09e645d85b8 registry.maas:5000/kolla/cinder-api:zed-ubuntu-jammy "dumb-init --single-…" 2 weeks ago Up 2 minutes (healthy) cinder_api
Setup the storpool volumes
(kolla-ansible-zed) define@kolla-deploy:~/kolla/config$ openstack volume type create --property volume_backend_name=storpool --property storpool_template='nvme' storpool-nvme +-------------+----------------------------------------------------------+ | Field | Value | +-------------+----------------------------------------------------------+ | description | None | | id | 14bce409-37bf-4b34-9f45-80628b9c772d | | is_public | True | | name | storpool-nvme | | properties | storpool_template='nvme', volume_backend_name='storpool' | +-------------+----------------------------------------------------------+ (kolla-ansible-zed) define@kolla-deploy:~/kolla/config$ openstack volume type list +--------------------------------------+---------------+-----------+ | ID | Name | Is Public | +--------------------------------------+---------------+-----------+ | 14bce409-37bf-4b34-9f45-80628b9c772d | storpool-nvme | True | | 62970386-e08a-49c9-8c09-d83902fdd8a2 | __DEFAULT__ | True | +--------------------------------------+---------------+-----------
setup the nova compute side
Globals.yaml updated
# storpool dp enable_cinder_backend_iscsi: "yes" enable_cinder_backend_lvm: "no" skip_cinder_backend_check: True cinder_volume_image: "registry.maas:5000/kolla/cinder-volume" cinder_volume_tag: "zed-ubuntu-jammy-storpool" cinder_volume_extra_volumes: - "/etc/storpool.conf:/etc/storpool.conf:ro" nova_compute_image: "registry.maas:5000/kolla/nova-compute" nova_compute_tag: "zed-ubuntu-jammy-storpool" nova_compute_extra_volumes: - "/etc/storpool.conf:/etc/storpool.conf:ro" # end storpool
Nova compute docker options
root@gpu-227:~# cat /etc/docker/daemon.json
{
"bridge": "none",
"exec-opts": [
"native.cgroupdriver=systemd"
],
"ip-forward": false,
"iptables": false,
"log-opts": {
"max-file": "5",
"max-size": "50m"
}
Notes from storpool on setting up systems
# As per the changes, I did the following:
# 1. Install the systemd-container package on the host.
apt-get install -y systemd-container.
# 2. Add an extra systemd drop-in to make systemd-machined start at boot.
mkdir -p /etc/systemd/system/systemd-machined.service.d,
# 3. Then add a file called 10-install.conf with the following contents:
[Install]
WantedBy=multi-user.target
# 4. Issue systemctl daemon-reload followed by
systemctl enable --now systemd-machined.
# 5. In case the nova_libvirt container is already deployed, restart it with
docker restart nova_libvirt.
# update this was in the notes from storpool - should sort the above I think
Extra Ansbile compute host variables in INI format:
docker_custom_config="{'exec-opts': ['native.cgroupdriver=systemd']}"
default_container_dimensions="{'cpuset_cpus': '0,2-15'}"
Update the compute nodes with the new container for nova-compute
kolla-ansible --configdir=$(pwd) -i inventory --limit=gpu-227 -t nova deploy