Integrating Travis CI and Codecov into a Python-based Project

A little background: over the last few months, I have been contributing to an open source organization FOSSASIA, where I’m working on a project called BadgeYaY.

It is a badge generator with a simple web UI to add data and generate printable badges in PDF. BadgeYay uses Flask, which is a Python based microframework.

I wanted to include a test coverage reporter in my continuous integration flow, which would ideally deploy to Heroku whenever my TravisCI build passes. For that, I decided to use codecov.io for integrating code coverage.

First, let’s take a look at what "code coverage” is, and then we shall move on to how to integrate Codecov with help of Travis CI.

What is Code Coverage ?

Simply put, code coverage is a measurement used to express the degree to which the source code is executed when a test suite runs. A program with higher code coverage means that the test suite has had more of the source code executed when it runs.

Thus, it implies that the source code has fewer chances of containing undetected bugs. We use three primary terms to describe source code lines executed.

• hit indicates that the given source code was executed by the test suite.
• partial indicates that the source code was not fully executed by the test suite and there are remaining branches that were not executed.
• miss indicates that the source code was not executed by the test suite.

Coverage is the ratio of hits / (hit + partial + miss). A code base that has 5 lines executed by tests out of 12 total lines will receive a coverage ratio of 41%. I’m not boasting, but as of publication time, BadgeYaY has 100% code coverage! 😎

Code coverage is like a tool for building tools, and is integrated to other tools, like Git and Travis CI. Source: XKCD

How does CodeCov help in Code Coverage ?

Codecov focuses on integration and promoting healthy pull requests. It delivers, or “injects” coverage metrics directly into the modern source code management workflow to promote more code coverage. This especially adds the convenience of checking coverage in pull requests where new features and bug fixes commonly occur.

We can change the configuration of how Codecov processes reports and expresses coverage information. Let us see how I configured it to suit BadgeYaY — by integrating it with Travis CI.

Now Codecov works great with Travis CI by using just one line:

bash < (curl -s https://codecov.io/bash)

BadgeYaY uses Python unit tests for assertion, with the help of Selenium for web browser automation.

Basically, that’s used for testing the code. I configured Codecov in such a way that it produces coverage reports based on the results of this testing, so I added the following to scripts in travis.yml

“scripts”: { 
— nosetests app/tests/test.py -v — with-coverage
}

Now I created a codecov.yml file that tells the configuration of generated reports after code coverage. Here’s the code:

codecov:
    notify:
    require_ci_to_pass: yes
    
coverage:
    precision: 2
    round: down
    range: “70…100”
    
status:
    project: yes
    patch: yes
    changes: no

comment:
    layout: “reach, diff, flags, files, footer”
    behavior: default
    require_changes: no
    

Here is some of the code for travis.ymlfrom the project repository of BadgeYaY, which integrates Codecov after successful builds.

Script:
- python app/main.py >> log.txt 2>&1 &
- nosetts app/tests/test.py -v — with-coverage
- python3 -m pyflakes

after_success:
- bash < (curl -s https://codecov.io/bash)

The other two scripts are not related to Codecov integration, but feel free to look them up if you’re interested.

Once all of this is set up, Codecov’s ready for action. Now, when anyone makes a pull request to BadgeYaY, Codecov will analyze it according to the above configuration and generate a report showing its code coverage.

With that, we have reached the end of our discussion on integrating Travis CI and Codecov into a Python-based project. I wrote this post as a solution to this issue in the BadgeYaY project. If you liked this post, consider having a look at my other work on GitHub 🙂.

Sources : codecov.io, Wikipedia

PS: I’m new to blogging, so constructive criticism is not only welcome, but very much wanted!