Skip to main content

Scalable Integration Suites

Welcome to the Scalable Integration Suites documentation!

When running a local handful of tests, execution time rarely matters. However, when migrating to a large-scale CI/CD environment with hundreds or thousands of integration tests running on every pull request, efficiency and pipeline compatibility become paramount.

Rumour is designed for high concurrency and exposes several flags designed specifically to scale horizontally and integrate seamlessly into automated pipelines.

1. High-Performance Parallel Execution (-p)​

By default, Rumour executes requests sequentially. However, because Rumour builds an internal Directed Acyclic Graph (DAG) of all dependencies, it knows exactly which requests can be executed at the same time without race conditions.

To drastically reduce your suite's execution time, use the --parallel (or -p) flag.

rumour run ./integration_suite/ -p --concurrency 50

How it works:​

  1. Rumour analyzes all .toml files.
  2. It groups requests into "Levels".
  3. All requests in Level 0 (no dependencies) are fired simultaneously, up to the limit set by --concurrency (default 20).
  4. Once a request's dependencies are satisfied, it is immediately dispatched.

2. CI/CD Pipeline Integration (--junit)​

Almost all modern CI/CD platforms (GitHub Actions, GitLab CI, Jenkins, CircleCI) use the JUnit XML format to render test dashboards and track failure trends over time.

Instead of writing custom parsers for Rumour's terminal output, simply pass the --junit flag to automatically generate a compliant XML report.

rumour run ./integration_suite/ --junit report.xml

Output Example (report.xml)​

<testsuites name="Rumour API Tests" tests="1" failures="0" time="0.002">
  <testsuite name="Main Workflow" tests="1" failures="0">
    <testcase name="/testing/advanced_examples/create_user.toml" classname="rumour.workflow" time="0.002"/>
  </testsuite>
</testsuites>

You can then upload this artifact directly into your pipeline's test reporter.

3. Ephemeral Resource Cleanup (-C)​

A common problem with integration testing is "database bloat". Test suites create hundreds of dummy users, orders, and sessions. If a suite crashes midway through, those records are permanently orphaned in the test environment.

Rumour provides an --auto-cleanup (or -C) flag designed for exactly this scenario.

rumour run ./integration_suite/ -C

How Auto-Cleanup works:​

  1. When a request returns 201 Created, Rumour tracks all variables extracted from its response (e.g., user_id).
  2. Rumour then searches for separate request files or nodes in the workspace directory containing delete or cleanup in their name (e.g., delete_user.toml or cleanup_user.toml).
  3. If a cleanup request is found that depends on the extracted variable, Rumour automatically executes it for each created resource instance, ensuring your test database remains pristine.

4. Stopping on First Failure (--stop-on-fail)​

If you are running a 5,000-request regression suite and node #4 fails, there's often no reason to wait 10 minutes for the other 4,996 requests to execute.

Use the --stop-on-fail flag to tell Rumour to abort the test suite immediately upon the first assertion failure, saving valuable CI minutes.

rumour run ./regression_suite/ --stop-on-fail