OpenStack:Rally

Using containerised Rally client

Rally needs OpenStack admin credentials, so from a machine storing the admin openrc file, run the following commands:

# docker run --rm -it -h rally-testing --entrypoint /bin/bash -v <your-admin-openrc-file>:/openrc:ro registry.vscaler.com:5000/rally-openstack:1.7.0
$ source /openrc
$ rally env create --from-sysenv --name=<arbitrary-name-for-environment>

The last command will also create a deployment object with the same name - run rally deployment list to see it.

Then, look into /home/rally/source/samples/tasks/scenarios/ for sample scenarios or write your own scenario. For example:

$ cat ~/test_boot_and_delete_instance.json
{
    "NovaServers.boot_and_delete_server": [
        {
            "args": {
                "flavor": {
                    "name": "m1.small"
                },
                "image": {
                    "name": "centos7-1907"
                },
                "force_delete": false
            },
            "runner": {
                "type": "constant",
                "times": 10,
                "concurrency": 2
            },
            "context": {
                "users": {
                    "tenants": 3,
                    "users_per_tenant": 2
                }
            }
        }
    ]
}

NOTE: All test scenarios are also available here: https://opendev.org/openstack/rally-openstack/src/branch/master/samples/tasks/scenarios.

List available plugins and scenarios:

$ rally plugin list --platform openstack

Finally, run the test:

$ rally task start /path/to/your/scenario.json

Visualising test results

First off, you'll need the UUID of the task you want to export results from:

$ rally task list --deployment <your-deployment-name>

Export to a HTML file:

$ rally task report <task-uuid> --out ~/output.html

Error creating thumbnail: File missing

Export to a JUnit XML (this file can then be used by Jenkins):

$ rally plugin list --platform openstack | grep junit
...
| TaskExporter         | junit-xml                          | default   | Generates task report in JUnit-XML format.           |
...
$ rally task export <task-uuid> --type junit-xml --to ~/output.xml

Rally-based test/validation suite

The code for the Rally-based test suite is available here:

https://gitlab.vscaler.com/mkarpiarz/rally-test-suite

Read the README file for details.

The general idea behind the suite is that the Client code (also known as the "Runner") has to be deployed and set up on a node with OpenStack credentials in the environment that one wishes to test. This Client then runs multiple test scenarios and sends a report to the Server, which then gathers and serves reports from multiple environments for viewing.

References

1. https://rally.readthedocs.io/en/latest/quick_start/tutorial.html
2. https://rally.readthedocs.io/en/latest/quick_start/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html
3. https://github.com/openstack/rally/tree/stable/0.12/samples/tasks/scenarios
4. https://opendev.org/openstack/rally-openstack
5. https://opendev.org/openstack/rally-openstack/src/branch/master/samples/tasks/scenarios
6. https://gitlab.vscaler.com/mkarpiarz/rally-test-suite