Python

This guide provides a walkthrough for a Python-based application and how to configure common CI tasks. If you’re new to Razorops please read our Tutorial and configuration guides first.

We will show how to download dependencies, run tests, generate artifacts or coverage reports using various popular tools in Python community. We also provide CI optimized container image for Python as well to use in your pipeline.

Specify Python version

Usually the first step before writing CI spec is to decide the language runtime and support for the version compatible with your application. We support most active versions of Python. You have the following options if you want to pin-down a version of Python -

• For Docker runner environment, you can specify a container image (razorci/python or official python) with tag.
• For Linux-VM environment, you can change the version (or install new version if not available) using pyenv.

Choosing version on Docker environment

You can control Python version using an appropriate docker tag in runner property. To know more about runner, see here.

3.92.7

tasks:
  py3:
    runner: razorci/python:3.9
    ...

tasks:
  py2:
    runner: razorci/python:2.7
    ...

Choosing version on Linux-VM environment

In Linux-VM build environment, we use pyenv to manage the various Python versions. You can find currently supported versions in toolbox reference. You can also install a different version using pyenv install <version> command if it's not supported by us.

Here are few commands to use, switch and install a version -

    # file: .razorops.yaml
    steps:
    - commands:
      # list installed versions
      - pyenv versions
      - python --version

      - pyenv global 3.10.5
      - python --version

      # install python 3.11
      - pyenv install --list ## list stable versions
      - pyenv install 3.11.3

      - pyenv global 3.11.3
      - python --version
        # Python 3.11.3

Note

Available python versions pre-installed via pyenv - 3.8.12, 3.9.13, and 3.10.5

Download dependencies

Downloading python third-party modules/dependencies from the internet can be time-consuming, you can greatly speed up this process by caching it using an unique key. For more information, see Caching dependencies.

In the below example we demonstrate a pipeline that restores a cache, executes pip install, and rebuilds the cache. Cache directory location will vary depending on your tool (pip, wheel or pipenv).

pippipenv

tasks:
  unit-test:
    runner: razorci/python:3.9
    steps:
    - checkout

    - cache/pull:
      - key: pip-cache-{{ checksum "requirements.txt" }}
    ...
    - run: pip install -r requirements.txt --user
    ...
    - cache/push:
      - key: pip-cache-{{ checksum "requirements.txt" }}
        paths:
        - ~/.cache/pip ## for pip

tasks:
  unit-test:
    runner: razorci/python:3.9
    steps:
    - checkout
    - cache/pull:
      - key: pipenv-cache-{{ checksum "Pipfile.lock" }}
    ...
    - commands: 
      - sudo pip install pipenv
      - pipenv install
    ...
    - cache/push:
      - key: pipenv-cache-{{ checksum "Pipfile.lock" }}
        paths:
        - ~/.local/share/virtualenvs/venv

Execute tests and generate reports

You can use the same command that you use locally with run property in steps and combine with various lint and coverage tools to generate reports. To run the tests, it is recommended that you use tox.

nosetestspytesttox

tasks:
  unit-test:
    runner: razorci/python:3.9
    steps:
    ...
    - commands: 
      - pip install -r requirements.txt --user

      # read more at https://nose.readthedocs.io/en/latest/plugins/xunit.html  
      - nosetests tests.py --with-coverage --cover-html --with-xunit
    ...

    # lint step using flake8, add it in requirements.txt
    - run: flake8 .
    ...

    - reports/html: 
        dir: cover # generated from line 10
    - reports/junit:
        paths: [nosetests.xml] # generated from line 10

tasks:
  unit-test:
    runner: razorci/python:3.9
    steps:
    ...
    - commands: 
      - pip install pytest pytest-cov
      - python -m pytest tests.py
    ...

## file: .razorps.yaml
tasks:
  unit-test:
    runner: razorci/python:3.9
    steps:
    ...
    - commands:
      - tox
    ...

## file: tox.ini
[tox]
envlist = py36,py37
[testenv]
deps = pytest
commands = pytest

Publish artifacts or results

You can upload artifacts to view after a workflow completes. For example, you may need to save log files, core dumps, test results, or screenshots. For more information, see "Persisting workflow data using artifacts."

The below example demonstrates how you can use the artifacts/push step to archive test results from running pytest. For more information, see the artifacts/push step.

tasks:
  unit-test:
    runner: razorci/python:3.9
    steps:
    ...
    - commands:
      - pip install pytest
      - python -m pytest tests.py --doctest-modules --junitxml=junit/test-results.xml
    ...
    - artifacts/push:
        name: test-results
        paths: [junit/test-results.xml]