OpenStack:Rally - Revision history

Mariusz: Add docs on running Tempest verification/validation with Rally

2020-11-03T14:44:42Z

Add docs on running Tempest verification/validation with Rally

Mariusz: /* Rally-based test/validation suite */ Add TODOs

2020-10-21T17:30:38Z

Rally-based test/validation suite: Add TODOs

Mariusz: /* References */ Add a link to docs describing task components

2020-10-16T11:43:30Z

References: Add a link to docs describing task components

Mariusz: /* Using containerised Rally client */ Add a note on creating a database for non-containerised Rally installs

2020-10-16T09:42:56Z

Using containerised Rally client: Add a note on creating a database for non-containerised Rally installs

Mariusz: Add a note on the Rally-based test suite

2020-10-16T09:35:39Z

Add a note on the Rally-based test suite

Mariusz: Update docs for Rally 1.7.0 (from 1.5.1) and newer.

2020-10-16T09:22:15Z

Update docs for Rally 1.7.0 (from 1.5.1) and newer.

Mariusz: /* Visualising test results */ Reduce number of spaces on the output from `rally plugin list`

2020-04-02T12:14:55Z

Visualising test results: Reduce number of spaces on the output from `rally plugin list`

Mariusz: /* Visualising test results */ Add a screenshot with HTML results

2020-04-02T12:13:04Z

Visualising test results: Add a screenshot with HTML results

Mariusz: /* Visualising test results */ No need to install junit-xml - just make sure junit-xml plugin is available. Also changes output paths to avoid permission issues when writing to the cwd.

2020-04-02T12:02:10Z

Visualising test results: No need to install junit-xml - just make sure junit-xml plugin is available. Also changes output paths to avoid permission issues when writing to the cwd.

Mariusz: Created page with "== Using containerised Rally client == Rally needs OpenStack admin credentials, so from a machine storing the admin openrc file, run the following commands: # docke..."

2020-04-02T11:56:48Z

Created page with "== Using containerised Rally client == Rally needs OpenStack admin credentials, so from a machine storing the admin openrc file, run the following commands: <nowiki> # docke..."

New page

== Using containerised Rally client ==
Rally needs OpenStack admin credentials, so from a machine storing the admin openrc file, run the following commands:

<nowiki>
# docker run --rm -it -h rally-testing --entrypoint /bin/bash -v <your-admin-openrc-file>:/openrc:ro registry.vscaler.com:5000/rally-openstack:latest
$ source /openrc
$ rally deployment create --fromenv --name=<arbitrary-name-for-deployment>
</nowiki>
Then, look into <code>/home/rally/source/samples/tasks/scenarios/</code> for sample scenarios or write your own scenario. For example:

<nowiki>
$ cat ~/test_boot_and_delete_instance.json
{
"NovaServers.boot_and_delete_server": [
{
"args": {
"flavor": {
"name": "m1.small"
},
"image": {
"name": "centos7-1907"
},
"force_delete": false
},
"runner": {
"type": "constant",
"times": 10,
"concurrency": 2
},
"context": {
"users": {
"tenants": 3,
"users_per_tenant": 2
}
}
}
]
}
</nowiki>

List available plugins and scenarios:

<nowiki>
$ rally plugin list --platform openstack
</nowiki>

Finally, run the test:

<nowiki>
$ rally task start /path/to/your/scenario.json
</nowiki>

== Visualising test results ==
First off, you'll need the UUID of the task you want to export results from:

<nowiki>
$ rally task list --deployment <your-deployment-name>
</nowiki>

Export to a HTML file:

<nowiki>
$ rally task report <task-uuid> --out output.html
</nowiki>

Export to a JUnit XML (this file can then be used by Jenkins):

<nowiki>
$ pip install junit-xml
$ rally task export <task-uuid> --type junit-xml --to output.xml
</nowiki>

== References ==

# https://rally.readthedocs.io/en/latest/quick_start/tutorial.html
# https://rally.readthedocs.io/en/latest/quick_start/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html
# https://github.com/openstack/rally/tree/stable/0.12/samples/tasks/scenarios

@@ Line 80: / Line 80: @@
 $ rally task export <task-uuid> --type junit-xml --to ~/output.xml
 </nowiki>
 == Rally-based test/validation suite ==
@@ Line 108: / Line 168: @@
 # https://gitlab.vscaler.com/mkarpiarz/rally-test-suite
 # https://rally.readthedocs.io/en/latest/task/index.html

← Older revision		Revision as of 17:30, 21 October 2020
Line 91:		Line 91:
	The general idea behind the suite is that the Client code (also known as the "Runner") has to be deployed and set up on a node with OpenStack credentials in the environment that one wishes to test. This Client then runs multiple test scenarios and sends a report to the Server, which then gathers and serves reports from multiple environments for viewing.		The general idea behind the suite is that the Client code (also known as the "Runner") has to be deployed and set up on a node with OpenStack credentials in the environment that one wishes to test. This Client then runs multiple test scenarios and sends a report to the Server, which then gathers and serves reports from multiple environments for viewing.

		+
		+	=== TODOs ===
		+
		+	* Move config parameters out of the code and into a config file.
		+	* Run Tempest validation tests from acceptance lists, like this one: https://refstack.openstack.org/api/v1/guidelines/2020.06/tests?type=required (docs: https://rally.readthedocs.io/en/latest/verification/index.html)
		+	* Find a method to add vScaler branding to HTML reports generated by Rally.
		+	* Deploy the server container on a dedicated instance (with a DNS record).

	== References ==		== References ==

← Older revision		Revision as of 11:43, 16 October 2020
Line 100:		Line 100:
	# https://opendev.org/openstack/rally-openstack/src/branch/master/samples/tasks/scenarios		# https://opendev.org/openstack/rally-openstack/src/branch/master/samples/tasks/scenarios
	# https://gitlab.vscaler.com/mkarpiarz/rally-test-suite		# https://gitlab.vscaler.com/mkarpiarz/rally-test-suite
		+	# https://rally.readthedocs.io/en/latest/task/index.html

@@ Line 79: / Line 79: @@
 $ rally task export <task-uuid> --type junit-xml --to ~/output.xml
 </nowiki>
 == References ==
@@ Line 87: / Line 98: @@
 # https://opendev.org/openstack/rally-openstack
 # https://opendev.org/openstack/rally-openstack/src/branch/master/samples/tasks/scenarios

@@ Line 72: / Line 72: @@
 $ rally plugin list --platform openstack | grep junit
 ...
-| TaskExporter         | junit-xml                                                      | default   | Generates task report in JUnit-XML format.                               |
+| TaskExporter         | junit-xml                          | default   | Generates task report in JUnit-XML format.           |
 ...
 $ rally task export <task-uuid> --type junit-xml --to ~/output.xml

@@ Line 8: / Line 8: @@
 </nowiki>
 The last command will also create a <code>deployment</code> object with the same name - run <code>rally deployment list</code> to see it.
 Then, look into <code>/home/rally/source/samples/tasks/scenarios/</code> for sample scenarios or write your own scenario. For example:
@@ Line 53: / Line 55: @@
 $ rally task start /path/to/your/scenario.json
 </nowiki>
 == Visualising test results ==

@@ Line 3: / Line 3: @@
   <nowiki>
-# docker run --rm -it -h rally-testing --entrypoint /bin/bash -v <your-admin-openrc-file>:/openrc:ro registry.vscaler.com:5000/rally-openstack:latest
+# docker run --rm -it -h rally-testing --entrypoint /bin/bash -v <your-admin-openrc-file>:/openrc:ro registry.vscaler.com:5000/rally-openstack:1.7.0
 $ source /openrc
-$ rally deployment create --fromenv --name=<arbitrary-name-for-deployment>
+$ rally env create --from-sysenv --name=<arbitrary-name-for-environment>
 </nowiki>
 Then, look into <code>/home/rally/source/samples/tasks/scenarios/</code> for sample scenarios or write your own scenario. For example:
@@ Line 38: / Line 40: @@
+}
 </nowiki>
 List available plugins and scenarios:
@@ Line 82: / Line 85: @@
 # https://rally.readthedocs.io/en/latest/quick_start/tutorial/step_1_setting_up_env_and_running_benchmark_from_samples.html
 # https://github.com/openstack/rally/tree/stable/0.12/samples/tasks/scenarios

@@ Line 64: / Line 64: @@
 $ rally task report <task-uuid> --out ~/output.html
 </nowiki>
 Export to a JUnit XML (this file can then be used by Jenkins):