Skip to content

YAML Specification

This document is a reference for the Razorops YAML configuration keys that are used in the razorops.yaml file.

version

version key defines the pipeline definition version. Currently only v2 is the supported values (default: v2).

global

global key defines settings common to all tasks in a pipeline. Currently it supports runner configuration and environment variables which are passed to each task. A task can override runner property as well.

global:
  runner:
    containers:
    - image: razorci/ruby:3
    - image: redis
  variables:
  - RACK_ENV=test

runner

runner key defines the execution environment of a workflow run.

We currently support two types of environments for running your tasks - Docker) and Linux (Ubuntu) VM.

Docker VM acts as a composable and flexible environment where you can customize it with container images (managed by your organization or community). Whereas, Linux VM is managed by Razorops engineers where most commonly used build tools and programming language runtimes are pre-installed. Both environment types have advantages and disadvantages compared to each other.

To choose Linux VM (Ubuntu) environment, you need to specify os_image property, otherwise we use Docker VM environment by default. An example -

global:
  runner:
    os_image: ubuntu

For Docker-based environment, you define a primary container (the first element in containers property) and other dependencies. All steps in the tasks are executed in the primary container. Razorops also provides CI-optimized container images (on dockerhub) for various languages and frameworks. An example -

global:
  runner:
    containers:
    - image: razorci/ruby:3
    - image: redis

Or if your pipeline needs only a docker image, you can provide a short-hand

global:
  runner: razorci/ruby:3

You can specify os_image and containers both which defines a VM environment to run the steps and the containers running in that environment. This is useful for running various service containers like for databases, caching services, etc. Here is an example -

global:
  runner: 
    # use ubuntu VM
    os_image: ubuntu
    # starts postgrses container in background and exposes the ports 
    # locally at localhost:5432
    containers:
    - image: postgres
      environment:
      - POSTGRES_PASSWORD=secret

tasks:  
  test: ## task name
    steps:
    # test connect
    - run: dockerize -wait tcp://localhost:5432
    ...
global:
  runner: 
    # use ubuntu VM
    os_image: ubuntu
    # starts redis container in background and exposes the ports 
    # locally at localhost:6379
    containers:
    - image: redis

tasks:  
  test: ## task name
    steps:
    # Ping test
    - run: (printf "PING\r\n";) | nc localhost 6379

You can specify multiple containers. These will be created/started before executing tasks[].steps and share the host network in the VM (localhost). global.runner property can be overridden by tasks[].runner property if you want to choose different environment for a task.

Note

Currently, we only support container/docker and Linux based environment. The support for Android, Windows, and Macosx based environment is coming soon.

os_image

It defines the name of VM image for the runner. Currently we provide ubuntu distribution (jammy/22.04 LTS). It's maintained by Razorops developers and updated at regular intervals. An example -

global:
  runner:
    os_image: ubuntu

containers

List of containers that make the environment for running your tasks.

For Docker environment, The first element is called a primary container responsible for executing the steps, while rest are started in the background sharing the host network, generally needed for running tests alongside database, caching or message broker services.

For VM environment, these containers are started in background and will acts as service containers though the steps will run on actual VM. For more details, see "How to use service containers".

Each container must define image property and can optionally have environment, command, and entrypoint properties.

  • environment defines the environment variables that are injected into the container process. They are defined as key, value pairs separated by = just like global variables and task variables.

  • command overrides the default CMD when the non-primary container starts.

  • options are passed as command line options when the non-primary container creates.

global:
  runner:
    containers:
    - image: razorci/ruby:3
      environment:
      - TOKEN=secret
    - image: mysql
      environment:
      - MYSQL_ROOT_PASSWORD=foobar
  ...

Note

command and options keys are not supported for the primary container.

variables

It defines environment variables injected into every task of a workflow. They can be interpolated with standard CI variables as well. They are defined as key, value pairs separated by =.

global:
  runner:
    ...
  variables:
  - RACK_ENV=test
  - SHORT_SHA=${CI_COMMIT_SHA:0:8}

tasks

A task is the sequence of steps to execute and determines the actions of your pipeline. This is the main object of the configuration with tasks key. Each task has a name, so will sometimes be referred to as a "named task". Tasks will run parallelly they are defined. tasks must have a name that should be unique within the .razorops.yaml file. The relationship between tasks is defined using depends property.

This section defines a list of steps that allow you to build, test and deploy your code. Steps are (by default) executed sequentially, in the order in which they are defined. If a step returns a non-zero exit code, the pipeline immediately aborts and returns a failure status.

Tip

Tasks have a maximum runtime of 1 hour. If your jobs are timing out, consider running some of them in parallel.

Here is an example -

tasks:
  build-image:
    steps:
    ....

  unit-tests:
    steps:
    ...

Each task consists of the tasks's name as a key, list of steps, and many more options. A name should be unique within a current tasks list. The most important key steps, defines list of execution units.

A task allows following parameters -

key Type Description
runner map Defines the build environment for the task. If not provided, global.runner will be used instead.
steps array List of steps with at-least one key which determines it's kind
variables array Environment variables in variables form
when string A conditional expression that omits the task when false
depends array List of tasks that this task depends on. It will only run after the named tasks have completed.

Here is an example of tasks with dependencies:

tasks:
  build:
    ...

  test1:
    depends: [build]
    ...

  test2:
    depends: [test1]
    ...

  deploy:
    depends: [test2]
    ...

The dependencies are defined by setting the depends: key as shown. The deploy: task will not run until the build and test1 and test2 tasks complete successfully. A task must wait until all upstream tasks in the dependency graph have run. So, the deploy task waits for the test2 task, the test2 task waits for the test1 task and the test1 task waits for the build task.

runner

It defines the environment for the specific task. Refer here for more details.

It's optional and should be used when you want to choose different build environment for a particular task as it overrides the global.runner property.

os_image

Defines the VM image for the specific task. Refer here for more details.

containers

List of containers to start for the specific task. Refer here for more details.

steps

Steps define an execution plan for a task in a serial order. They are run sequentially till at-least one of the step fails. Razorops has several inbuilt steps that can be included in your pipeline -

Kind Description
run Execute a command in the shell
commands Execute multiple command(s) in the shell
checkout Clone the source code repository using git
docker/build Build and optionally push your Docker image
docker/push Push a docker image into container registry
cache/pull Download objects from the caching storage with given key(s)
cache/push Upload objects to the caching storage with a key
artifacts/pull Download task artifacts
artifacts/push Upload task artifacts
workspace/persit Persist files/directories for downstream tasks (create a layer context)
workspace/attach Restore all layer contexts
reports/html Save HTML coverage reports
reports/junit Save and upload junit files for Insights

You can also specify following properties for a step -

  • if - defines a conditional expression to execute the step only if evaluated to true.
  • when - defines a condition to run the step based on the task exit status. It should be one of on_success, on_fail and always (defaults to on_success).
  • allow_failure - If true (boolean) the step failure won't abort the pipeline.
  • variables - defines array of environment variables for the step. They are defined as key,value pairs separated by =.
  • background - If you want to run the step in background and don't care about the exit status of the step. Generally used to start a server process in background without blocking the execution. Refer here for more details.
  • parallelism - defines a set of tasks with same steps that can be run in parallel. The parallelism property expects integer value larger than 1 (max: 8). For more details, see "Parallelism".

  • timeout* - defines a timeout in seconds (integer) before the runner kills the step.

Note

Step properties marked as * above, are not supported as of now, will be coming soon.

    steps:
    - checkout
    - docker/build:
        image: example.com/demo
      if: branch == 'master'
      allow_failure: true
    ...
    - run: make clean
      when: always
    ...
    - run: server -p 8080
      background: true
    ...

run

This step is designed so you can execute one or more shell commands. The task fails if any of the commands fail or return non-zero exit code. It's an alias for commands step.

tasks:
  unit-tests:
    runner: ruby:2.6
    steps:
    - checkout
    - run: bundle install
    - commands:
        - bundle check
        - bundle exec rspec

commands

Similar to run step, it accepts a list of commands to execute in the current environment/shell.

checkout

checkout step will perform a git clone of source code repository. Normally this step should be provided at the start of steps in a task which needs all files from the repository. Afterward, you can persist the required files to downstream tasks.

tasks:
  unittest:
    steps:
    - checkout
    - run: echo Hello

docker/build

Before deploying anything, you will most of the time want Razorops to build your Docker images. For this you need to use the docker/build task, which is one of the inbuilt step.

The only mandatory parameter of build is the image you want to build:

tasks:
  build-image:
    steps:
    - checkout
    - docker/build:
       image: razorci/docs
       push: true
       tags: ["${CI_COMMIT_SHA}", "latest"]
       dockerfile: Dockerfile
       context: .
       args:
       - API_TOKEN=foobar

docker/build step allows following parameters -

key Type Default Description
image * String The name of docker image to build
tags Array ["latest"] The list of tags for docker images
push Boolean false push the image to a docker registry
args Array Docker build arguments (--build-args)
args-env Array Docker build arguments to be used from secrets configured in Dashboard
dockerfile String ./Dockerfile Path of dockerfile relative to root of your source repo
context String . Docker context (directory)
target String target stage in a multistage build (build will run until this stage)

Tip

Always think carefully before pushing your images to a public Docker repository. If you are using a private Git repository or your build contains secrets (e.g. Github token, database passwords) then you need to ensure that you use a private Docker repository. Docker Hub and Quay.io both offer private repositories as part of their subscription plans.

docker/push

Push a built image to a remote Docker registry with one or more tags. It supports standard Docker registries, GCR and ECR.

push task supports following parameters:

Key Type Default Description
image * string Docker image (public or private)
tags array Multiple tags under which to push the image

Tip

You can optionally push docker image with docker/build task as well with push: true.

cache/push

Generates and stores a cache of a file or directory of files such as dependencies or source code in our object storage. Later/downstream jobs can restore this cache.

The cache for a specific key is immutable and cannot be changed once written. For more details, see "Caching dependencies".

Key Type Description
key * string Unique identifier for the cache
paths * array List of folders or files to cache

Here is an example -

tasks:
  save-npm:
    steps:
    ...
    - cache/push:
        key: npm-cache-{{ checksum "package-lock.json" }}
        paths:
          - node_modules

cache/pull

It restores a previously saved cache based on a key. Cache needs to have been saved first for this key like in save-npm step. For more details, see "Caching dependencies".

Key Type Description
key * string Unique identifier of cached object
fallback_key string Unique identifier of cached object

Here is an example -

tasks:
  restore-npm:
    steps:
    ...
    - cache/pull: 
        key: npm-cache-{{ checksum "package-lock.json" }}
        fallback_key: npm-cache-

artifacts/push

Generates and stores an artifact of a file or directory of files (for example, distribution zip files or jar files) in our object storage so that they can be downloaded later.

The cache for a specific key is immutable and cannot be changed once written.

Key Type Description
name * string Artiface name
paths * array List of folders or files to include

Here is an example -

tasks:
  save-npm:
    steps:
    ...
    - artifact/push:
        name: binaries
        paths:
          - bin

artifacts/pull

It downloads a previously saved artifact within the pipeline or different pipeline in current workspace. Artifact needs to have been pushed first.

Key Type Description
name * string Unique identifier of artifact
pipeline string Pipeline identifier

Here is an example -

tasks:
  restore-npm:
    steps:
    ...
    - artifacts/pull:
        name: binaries

workspace/persist

Persist a collection of files or directories such as build jars or compiled artifacts for downstream tasks. For more details, see "Working with layers".

The cache for a specific key is immutable and cannot be changed once written.

Key Type Description
paths * array List of folders or files to persist

Here is an example -

tasks:
  save-npm:
    steps:
    ...
    - workspace/persist:
        paths: [node_modules]

workspace/attach

It downloads saved workspace layers from upstream tasks within a workflow run. Layers needs to have been pushed first. For more details, see "Working with layers".

Here is an example -

tasks:
  restore-npm:
    steps:
    ...
    - workspace/attach

reports/html

If you are running unit/integration tests which generate coverage report or any html files which you need to introspect after the completion of a workflow, you can save them using this step.

Here is an example -

tasks:
  restore-npm:
    steps:
    ...
    - reports/html:
        dir: coverage
        index: index.html

reports/junit

You can configure a step to use JUnit test reports, and Razorops will display a report in Insight section in the piepline in dashboard. It helps to identify the most failing and slowest test without having to check the entire log.

Here is an example -

tasks:
  restore-npm:
    steps:
    ...
    - reports/junit:
      - spec/*.xml
    ...
    # or
    - reports/junit:
        paths: [spec/1.xml, spec/2.xml]

when

You can filter the execution of each task by using the when key in the YAML file. This can be done with an expression, as you can see in the following sample configuration:

tasks:
  build-image:
    steps:
    ...
    when: branch == 'master'

The expression has access to the workflow related context objects branch, tag, and event. Read here for the more details.

depends

Tasks can be define in a direct acyclic graph to build relationships between them and finish in quickest manner possible. Such dependencies are defined between tasks using depends property. This lets you run some tasks without waiting for other ones.

Here is an example

tasks:
  a:
    steps:
      ...

  b:
    depends: [a]

  c:
    depends: [a]

In this example, the task b and c will be executed in parallel as soon as a completes.

trigger

By default, Razorops will create a new workflow on every code change to the your repository. You can limit the branches or tags that you want to run with trigger property.

when

It helps to combine if and unless property. You can also define a shorthand to infer the if expression.

if

Razorops will trigger a new workflow run if result of if statement is evaluated to be true. For more details, see "Conditional expressions".

trigger:
  ## triggers pipeline only on feature branches and on tags (mind the prefix)
  when:
    if: branch =~ '^feature-*' || tag =~ '^v.*'

Or you can define a shorthand -

trigger:
  ## triggers pipeline only on feature branches and on tags (mind the prefix)
  when: branch =~ '^feature-*' || tag =~ '^v.*'

unless

Razorops will not trigger a new workflow run if result of unless statement is evaluated to be true. For more details, see "Conditional expressions".

trigger:
  ### triggers pipeline on non-master branches
  when:
    unless: branch == 'master'

schedules

It allows you to trigger a new workflow at a scheduled time periodically. You can specify multiple schedules with different frequency. For example -

trigger:
  schedules:
  - name: daily
    cron: "15 12 * * *"
    branch: master

Each item in schedules key supports the following properties -

  • name - defines an unique name for the schedule in the pipeline
  • branch- defines the branch to run the new workflow against.
  • cron - It uses cron patterns to define the shcedule to repeat. You can use the crontab.guru tool to check that a CRON pattern matches the intended schedule.

For more information on schedules, please follow Scheduled workflows section.

Note

Scheduled workflows run on the default or provided branch. The shortest interval you can run scheduled workflows is once every 5 minutes.

workflows

workflows defines multiple possible execution paths (workflow runs) with their run order of tasks (defined under tasks property) to model complex pipelines. For more details, check "Workflows and Tasks". Here is an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
tasks:
  ## define build, test, publish and deploy tasks
  ...

workflows:
- name: test-build
  tasks:
  - build
  - test
  - publish:
      depends: [build, test]

- name: test-build-deploy
  when: branch == "master"
  tasks:
  - build
  - test
  - publish:
      depends: [build, test]
  - deploy:
      depends: [publish]

Each element in workflows key supports following properties:

  • name - defines an unique name for the workflow run.
  • tasks - It defines the run order of the tasks in the workflow run. Each element in this array should be defined under tasks section (refer line 1 in example). Checkout tasks to know more about how to define a task in Pipeline YAML.
  • tasks[].depends - List of tasks which should be completed before this task. The dependencies should be included in the array (refer line 8,9 in example) and defined under tasks (refer line 1 in example)
  • when - You can skip the execution of the workflow by using the when key. This can be done with a conditional expression. For more details, check "Conditional Expressions".

workflows key is optional and only needed to build a complex CI process. Alternatively, you can also define a DAG of tasks using depends key under tasks.