|Dev cycle||132 days|
This release contains new features, new 42 plugins, 90 bug fixes, various code and API improvements.
New Features & API changes¶
Improved installation script
- Add parameters:
- --develop parameter to install rally in editable (develop) mode
- --no-color to switch off output colorizing useful for automated output parsing and terminals that don't support colors.
- Puts rally.conf under virtualenv etc/rally/ so you can have several rally installations in virtualenv
- Many fixes related to access of different file, like: rally.conf, rally db file in case of sqlite
- Update pip before Rally installation
- Fix reinstallation
- Add parameters:
Separated Rally plugins & framework
Now plugins are here: https://github.com/openstack/rally/tree/master/rally/plugins
Plugins are as well separated common/* for common plugins that can be use no matter what is tested and OpenStack related plugins
New Rally Task framework
All plugins has the same Plugin base: rally.common.plugin.pluing.Plugin They have the same mechanisms for: discovering, providing information based on docstrings, and in future they will use the same deprecation/rename mechanism.
Some of files are moved:
rally/benchmark -> rally/task
This was done to unify naming of rally task command and actually code that implements it.
rally/benchmark/sla/base.py -> rally/task/sla.py
rally/benchmark/context/base.py -> rally/task/context.py
rally/benchmark/scenarios/base.py -> rally/task/scenario.py
rally/benchmark/runners/base.py -> rally/task/runner.py
rally/benchmark/scenarios/utils.py -> rally/task/utils.py
This was done to:
- avoid doing rally.benchmark.scenarios import base as scenario_base
- remove one level of nesting
- simplify framework structure
Some of classes and methods were renamed
- context.context() -> context.configure()
- scenario.scenario() -> scenario.configure()
- Introduced runner.configure()
- Introduced sla.configure()
This resolves 3 problems:
Unifies configuration of different types of plugins
Simplifies plugin interface
- Looks nice with new modules path:
>>> from rally.task import scenario >>> @scenario.configure()
Atomic Actions were changed:
New rally.task.atomic module
This allow us in future to reuse atomic actions in Context plugins
rally.benchmark.scenarios.base.AtomicAction -> rally.task.atomic.ActionTimer
rally.benchmark.scenarios.base.atomic_action() -> rally.task.atomic.action_timer()
Context plugins decide how to map their data for scenario
Now Context.map_for_scenario method can be override to decide how to pass context object to each iteration of scenario.
Samples of NEW vs OLD context, sla, scenario and runner plugins:
# Old from rally.benchmark.context import base @base.context(name="users", order=100) class YourContext(base.Context): def setup(self): # ... def cleanup(self): # ... # New from rally.task import context @context.configure(name="users", order=100) class YourContext(context.Context): def setup(self): # ... def cleanup(self): # ... def map_for_scenario(self): # Maps context object to the scenario context object # like context["users"] -> context["user"] and so on.
# Old Scenario from rally.benchmark.scenarios import base from rally.benchmark import validation class ScenarioPlugin(base.Scenario): @base.scenario() def some(self): self._do_some_action() @base.atomic_action_timer("some_timer") def _do_some_action(self): # ... # New Scenario from rally.task import atomic from rally.task import scenario from rally.task import validation # OpenStack scenario has different base now: # rally.plugins.openstack.scenario.OpenStackScenario class ScenarioPlugin(scenario.Scenario): @scenario.configure() def some(self): self._do_some_action() @atomic.action_timer("some_action") def _do_some_action(self): # ...
## Old from rally.benchmark.runners import base class SomeRunner(base.ScenarioRunner): __execution_type__ = "some_runner" def _run_scenario(self, cls, method_name, context, args) # Load generation def abort(self): # Method that aborts load generation ## New from rally.task import runner @runner.configure(name="some_runner") class SomeRunner(runner.ScenarioRunner): def _run_scenario(self, cls, method_name, context, args) # Load generation def abort(self): # Method that aborts load generation
# Old from rally.benchmark import sla class FailureRate(sla.SLA): # ... # New from rally.task import sla @sla.configure(name="failure_rate") class FailureRate(sla.SLA): # ...
Rally Task aborted command
Finally you can gracefully shutdown running task by calling:
rally task abort <task_uuid>
Rally CLI changes
- [add] rally --plugin-paths specify the list of directories with plugins
- [add] rally task report --junit - generate a JUnit report This allows users to feed reports to tools such as Jenkins.
- [add] rally task abort - aborts running Rally task when run with the --soft key, the rally task abort command is waiting until the currently running subtask is finished, otherwise the command interrupts subtask immediately after current scenario iterations are finished.
- [add] rally plugin show prints detailed information about plugin
- [add] rally plugin list prints table with rally plugin names and titles
- [add] rally verify genconfig generates tempest.conf without running it.
- [add] rally verify install install tempest for specified deployment
- [add] rally verify reinstall removes tempest for specified deployment
- [add] rally verify uninstall uninstall tempest of specified deployment
- [fix] rally verify start --no-use --no-use was always turned on
- [remove] rally use now each command has subcommand use
- [remove] rally info
- [remove] rally-manage tempest now it is covered by rally verify
New Rally task reports
- New code is based on OOP style which is base step to make plugable Reports
- Reports are now generated for only one iteration over the resulting data which resolves scalability issues when we are working with large amount of iterations.
- New Load profiler plot that shows amount of iterations that are working in parallel
- Failed iterations are shown as a red areas on stacked are graphic.
Non backward compatible changes¶
- [remove] rally use cli command
- [remove] rally info cli command
- [remove] --uuid parameter from rally deployment <any>
- [remove --deploy-id parameter from: rally task <any>, rally verify <any>, rally show <any>
Specs & Feature requests¶
[feature request] Explicitly specify existing users for scenarios
[feature request] Improve install script and add --unistall and --version
[feature request] Allows specific repos & packages in install-rally.sh
[feature request] Add ability to capture logs from tested services
[feature request] Check RPC queue perfdata
[spec] Refactoring Rally cleanup
[spec] Consistent resource names
Add context for setting up Manila quotas: shares, gigabytes, snapshots, snapshot_gigabytes, share_networks
Context for share networks that will be used in case of usage deployment with existing users. Provided share networks via context option "share_networks" will be balanced between all share creations of scenarios.
Context to create LBaaS-v1 resources
Allows image customization using side effects of a command execution. E.g. one can install an application to the image and use these image for 'boot_runcommand_delete' scenario afterwards.
Context that creates servers using EC2 api
This context lets you use existing networks that have already been created instead of creating new networks with Rally. This is useful when, for instance, you are using Neutron with a dumb router that is not capable of creating new networks on the fly.
[remove] max_failure_rate - use failure_rate instead
90 bugs were fixed, the most critical are:
- Many fixes related that fixes access of rally.conf and DB files
- Incorrect apt-get "-yes" parameter in install_rally.sh script
- Rally bash completion doesn't exist in a virtualenv
- Rally show networks CLI command worked only with nova networks
- RPS runner was not properly generating load
- Check is dhcp_agent_scheduler support or not in network cleanup
- NetworkContext doesn't work with Nova V2.1
- Rally task input file was not able to use jinja2 include directive
- Rally in docker image was not able to
- Rally docker image didn't contain samples
- Do not update the average duration when iteration failed
Add plugin reference page
Rally Plugins Reference page page contains a full list with
Add maintainers section on project info page
Rally Maintainers section contains information about core contributors of OpenStack Rally their responsibilities and contacts. This will help us to make our community more transparent and open for newbies.
Added who is using section in docs
Many small fixes