Running tests in parallel¶
Though we support modeling a pipeline into a DAG structure to shorten the overall build time but sometimes it's not enough like executing a large test suite can slowdown the entire pipeline. You could use parallelism to split your test suite across a few jobs and save time.
To create multiple instances of a task, you can use
parallelism property and split your test suite. In order to run it as fast it possible, we need to ensure the steps will run a subset of test suite in a way that all commands complete at the same time to avoid a slow pipeline bottleneck. Razorops also injects two special environment variables to help you dynamically split the test suite -
CI_NODE_INDEXthe CI node index (between
CI_NODE_TOTALthe total number of CI nodes (
To split the test suite, you can use various tools and libraries as per your language framework like
parallel_tests for ruby.
tasks: e2e-tests: parallelism: 6 steps: - checkout - run: yarn test deploy: depends: [e2e-tests]
The above template will create 6 independent tasks with the same set of steps running parallely. Any task depending on the parallel task will run when all instances of parallel tasks are completed.
| e2e-tests | -> |e2e-tests-0| |e2e-tests-1| | ...... | ----> | deploy| |e2e-tests-5| └---✔ ci ├-✔ e2e-tests | └-·-✔ e2e-tests-0 32s | ├-✔ e2e-tests-1 30s | ├-✔ e2e-tests-2 23s | ├-✔ e2e-tests-3 32s | ├-✔ e2e-tests-4 30s | └-✔ e2e-tests-5 23s └-✔ deploy 19s