Working with test reports

Test reports help you to better understand test failures, code coverage, and various metadata around tests like execution time, flakiness etc. Code coverage is a metric that gives you an overview of how much of the program is covered with tests.

If your reporting tool produces reports in HTML or JUnit format, you can use reports/html or reports/junit step in pipeline. These reports are saved for every build and can be viewed at any point in time in the dashboard. You can generate these reports as part of test execution and have Razorops collect the output.

An example -

tasks:
  unit-test:
    runner: razorci/ruby
    steps:
    ...
    # generate a JUnit XML and coverage report
    - run: bundle exec rspec --format RspecJunitFormatter --output junit.xml
    ...
    - reports/html:
        dir: coverage
    - reports/junit:
        paths: [junit.xml]

There are two types of reports supported in Razorops -

JUnit XML

You can configure a step to use JUnit test reports, and Razorops will display a report in Insight section in the pipeline in dashboard. It helps to identify the most failing and slowest test without having to check the entire log.

reports/junit supports the following options -

• paths - List of XML file(s) or glob expression. The paths must exist and have valid XML format. In case of glob expression, we will try to recursively find matched files and collect them.

HTML

If you are running unit/integration tests which generate coverage report or any html files which you need to introspect after the completion of a workflow, you can save them using this step. Razorops will display a report in Reports section in the piepline in dashboard.

reports/html supports the following options -

• dir - The path of the directory where HTML files can be found.
• index - The index file to be served when we show the reports in the dashboard. By default, it looks for <dir>/index.html.

Ruby example

Use the following snippet in your pipeline -

# Uses rspec-junit-formatter gem (https://github.com/sj26/rspec_junit_formatter) to generate Junit XML
  steps:
  - checkout
  - commands:
    - bundle install
    - bundle exec rspec --format progress --format RspecJunitFormatter --out rspec.xml
  - reports/junit:
    - rspec.xml

Go example

Use the following snippet in your pipeline -

## Use https://github.com/jstemmer/go-junit-report to generate a JUnit report format XML file with go  
  steps:
  - checkout
  - commands:
    - go install github.com/jstemmer/go-junit-report@latest
    - go test -v 2>&1 | go-junit-report -set-exit-code > report.xml
  - reports/junit:
    - report.xml

Java example

GradleMaven

...
steps:
- checkout
- run: gradle test
- reports/junit:
    - build/test-results/test/**/TEST-*.xml

...
# Parses Surefire and Failsafe XML reports
steps:
- checkout
- run: mvn verify
- reports/junit:
    - target/surefire-reports/TEST-*.xml
    - target/failsafe-reports/TEST-*.xml