CMake Coverage Example: with GitHub Actions and codecov.io

Rodney Lab - Jan 29 - - Dev Community

🧑🏽‍💻 CMake Coverage Example

In this post, we see a CMake coverage example using GitHub Actions. We build the project and run unit tests on each git push, before uploading the latest code coverage results to codecov.io. This is all working on a CMake C++ project. Using gcov, genhtml and lcov locally, it is possible to generate an HTML coverage report, with graphs. However, here we automate the process, running tests remotely. This has added advantages. For example, you might configure your repo to reject pull requests that cause the test coverage to drop below a target percentage.

🧱 What we are Building

CMake Coverage Example: Screenshot shows codecov.io console. In the main view, a graph shows around 50% coverage over January. To the right, a donut represents this information with red and green segments.

We will use an arkanoid-clone project that I set up for testing. This is a clone of the classic arcade game built using CMake and SFML. The repo itself is based on a YouTube tutorial, and you can find a link to the original tutorial in the arkanoid-clone repo README. You might also want to reference the repo for any extra useful details missing here — do let me know, though, if there is anything, extra, I could add here to make the article clearer.

Let’s start by adding Catch2 tests to the repo. We assume you have a CMake repo ready to add tests, too. If you don’t, there is a fantastic JetBrains tutorial on adding Catch2 unit tests in CMake, which you can set up quickly as a toy project.

☑️ Setting up CMake Unit Tests

I will add Catch2 tests in a new Catch_tests directory, and need to include the new directory at the bottom of the main CMakeLists.txt file:

# include test folder towards the bottom of CMakeLists.txt
option(RUN_UNIT_TESTS "Run Catch2 unit tests" ON)
if(RUN_UNIT_TESTS)
    enable_testing()
    add_subdirectory(Catch_tests)
endif()
Enter fullscreen mode Exit fullscreen mode

Then, I create Catch_tests/CMakeLists.txt:

include(FetchContent)
FetchContent_Declare(
    Catch2
    GIT_REPOSITORY https://github.com/catchorg/Catch2.git
    GIT_TAG v3.5.0)
FetchContent_MakeAvailable(Catch2)

add_executable(Catch_tests_run BallTests.cpp)

target_link_libraries(
    Catch_tests_run
    PRIVATE Ball_lib
            Brick_lib
            Paddle_lib
            arkanoid_compiler_flags)
target_link_libraries(Catch_tests_run PRIVATE Catch2::Catch2WithMain)
target_include_directories(Catch_tests_run PUBLIC "${PROJECT_SOURCE_DIR}/src")

include(Catch)
catch_discover_tests(Catch_tests_run)
Enter fullscreen mode Exit fullscreen mode

This fetches the Catch2 dependency from GitHub, adds a test executable and finally, sets up Catch2 to discover the tests.

As an example test, you might have something like src/Catch_tests/BallTests.cpp:

#include "Ball.h"

#include <SFML/Graphics.hpp>
#include <catch2/catch_test_macros.hpp>

TEST_CASE("BallCorrectlyInitialised", "[BallTests]")
{
    const Ball ball{256, 512};
    const sf::Color ball_colour{ball.shape.getFillColor()};

    CHECK(ball_colour == sf::Color::Red);
}
Enter fullscreen mode Exit fullscreen mode

See the tutorial, mentioned above, for more on creating Catch2 tests and setting them up in CMake.

⛺️ Adding Coverage Tests

To start, download the CMake lcov Coverage module and add it to a new cmake folder in the root directory of your project:

function(add_coverage_target exclude)

    find_program(GCOV gcov)
    if (NOT GCOV)
        message(WARNING "program gcov not found")
    endif()

    find_program(LCOV lcov)
    if (NOT LCOV)
        message(WARNING "program lcov not found")
    endif()

    find_program(GENHTML genhtml)
    if (NOT GENHTML)
        message(WARNING "program genhtml not found")
    endif()

    if (LCOV AND GCOV AND GENHTML)
        set(covname cov.info)
        add_compile_options(-fprofile-arcs -ftest-coverage)
        add_link_options(--coverage)
        add_custom_target(cov DEPENDS ${covname})
        add_custom_command(
            OUTPUT  ${covname}
            COMMAND ${LCOV} -c -o ${covname} -d . -b . --gcov-tool ${GCOV}
            COMMAND ${LCOV} -r ${covname} -o ${covname} ${exclude}
            COMMAND ${LCOV} -l ${covname}
            COMMAND ${GENHTML} ${covname} -output coverage
            COMMAND ${LCOV} -l ${covname} 2>/dev/null | grep Total | sed 's/|//g' | sed 's/Total://g' | awk '{print $1}' | sed s/%//g > coverage/total
        )
        set_directory_properties(PROPERTIES
            ADDITIONAL_CLEAN_FILES ${covname}
        )
    else()
        message(WARNING "unable to add target `cov`: missing coverage tools")
    endif()

endfunction()
Enter fullscreen mode Exit fullscreen mode

Then, update the main CMakeLists.txt to find and use this, adding these lines towards the top of the file:

set(CMAKE_MODULE_PATH "${CMAKE_MODULE_PATH};${PROJECT_SOURCE_DIR}/cmake")
include(coverage)
add_coverage_target("*/Catch_tests/*")
Enter fullscreen mode Exit fullscreen mode

The argument of the add_coverage_target function is passed to lcov and gcov as an exclude path.

That is all you need to run the coverage tests. We will convert the data to XML for codecov.io later, using gcovr.

CMake Coverage Example: GitHub Test & Coverage Action

Next we need a GitHub action. This runs on every pull request submission. In our case, we use it to build the project, run tests and then generate a coverage report in XML format ready for codecov.io. Create a GitHub workflow flow in .github/workflows/ubuntu.yml to do this:

name: Ubuntu CI Test
on:
  push:
    branches: [main, master, dev]
  pull_request:
    branches: [main, master, dev]
permissions:
  contents: read
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2.7.0
      - name: install
        run: sudo apt-get update && sudo apt-get install lcov #libxrandr-dev libxcursor-dev libudev-dev libopenal-dev libflac-dev libvorbis-dev libgl1-mesa-dev libegl1-mesa-dev freeglut3-dev # OpenGL dependencies only needed for SFML
      - name: configure
        run: |
          cmake -H. -Bbuild -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Debug -DENABLE_COVERAGE=On
      - name: building
        run: |
          cmake --build build --config Debug --target Catch_tests_run
      - name: run unit tests
        working-directory: ./build/bin
        run: |
          ./Catch_tests_run
      - name: generate coverage
        working-directory: ./build
        run: |
          make cov
      - name: Install gcovr
        run: |
          pip install -r requirements.txt  --require-hashes
      - name: Generate JSON coverage report
        working-directory: ./build
        run: |
          gcovr -r .. . --branches --cobertura > coverage.xml
      - name: Upload coverage reports to Codecov
        uses: codecov/codecov-action@eaaf4bedf32dbdc6b720b63067d99c4d77d6047d # v3
        env:
          CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
Enter fullscreen mode Exit fullscreen mode

I use git commit hashes instead of repo tags or version numbers for added security (in line 13, for example). In line 15 you can install gcovr and lcov as well as any extra dependencies needed to build your project.

Lines 16 to 21 will configure, then build the project in the remote environment, after a pull request. Then, the code in lines 22 to 25 will run the Catch2 unit tests. If we broke something in the pull request, and the unit tests fail, the whole pull request is rejected: exactly what we are looking for.

In lines 26-29, we generate the coverage report in a text format. I had most success with codecov.io working in the XML format. We can convert the report, running:

gcovr -r .. . --branches --cobertura > coverage.xml
Enter fullscreen mode Exit fullscreen mode

Which is exactly what we do in lines 33-36, because our working directory will be build/ we adjust the path passed to gcovr. You can find more details on gcovr commands in the docs.

The final lines are where we upload the XML coverage report to codecov.io. You will need to obtain a codecov token and add this to your GitHub repo. To get a token, go to https://app.codecov.io/gh/YOUR-GITHUB-ACCOUNT, log in with GitHub and select the repo you want to add coverage for. Then, follow instructions on adding a repository secret. That will cover step one, displayed, and we have tackled step 2 above!

CMake Coverage Example: Screenshot shows codecov.io console. The title reads, Let's get your repo covered. Below, there are two steps outlines, adding a repository secret and adding codecov to your Git Hub workflow.

Python requirements.txt

Finally, add a requirements.txt file in the project root directory, which will be used in line 32, above, to install gcovr (and the lxml dependency):

gcovr==6.0 \\
  --hash=sha256:2e52019fdb76c6e327f48c2a2d8555fb5e362570b79cc74c5498804d1ce54a60
lxml==5.1.0 \\
  --hash=sha256:22b7ee4c35f374e2c20337a95502057964d7e35b996b1c667b5c65c567d2252a
Enter fullscreen mode Exit fullscreen mode

💯 CMake Coverage Example: Testing your Work

Next time you merge a pull request, after a short delay, you will see a coverage report on the codecov.io console, for your project.

CMake Coverage Example: Screenshot shows codecov.io console. In the main view the source code lines are highlighted green, if they were covered by the tests, and red and yellow are used for uncovered and partially covered lines. At the top, a summary shows the 3-Month average of 49% coverage, for the file.

CMake Coverage Example: Screenshot shows a panel on the Git Hub pull request screen.The title reads Codecov Report. A sort explanation is followed by a plain text table, summarizing the coverage results.

🙌🏽 CMake Coverage Example: Wrapping Up

In this CMake coverage example post, we saw a complete example of adding coverage tests to a CMake repo. We used GitHub actions to run a test coverage report, and push it to codecov.io. More specifically, we saw:

  • how you can use run Catch2 unit test in GitHub actions;
  • the codecov.io setup process for working in GitHub; and
  • how to run code coverage test in CI and push the result to codecov.io.

I hope you found this useful. If you need additional pointers, take a look at the arkanoid-clone repo mentioned at the start of the post, which fully implements the coverage approach discussed here. Do let me know, though, if anything here is not clear, or would benefit for further clarification.

🙏🏽 CMake Coverage Example: Feedback

If you have found this post useful, see links below for further related content on this site. Let me know if there are any ways I can improve on it. I hope you will use the code or starter in your own projects. Be sure to share your work on X, giving me a mention, so I can see what you did. Finally, be sure to let me know ideas for other short videos you would like to see. Read on to find ways to get in touch, further below. If you have found this post useful, even though you can only afford even a tiny contribution, please consider supporting me through Buy me a Coffee.

Finally, feel free to share the post on your social media accounts for all your followers who will find it useful. As well as leaving a comment below, you can get in touch via @askRodney on X (previously Twitter) and also askRodney on Telegram. Also, see further ways to get in touch with Rodney Lab. I post regularly on Deno as well as Search Engine Optimization among other topics. Also, subscribe to the newsletter to keep up-to-date with our latest projects.

Image
Some Free Icon Libraries </>

webdev, css
Image
20 Trailblazers Setting The Standard In Lawyer Injury Accident
Image
24 Hours For Improving Lost Car Key
Image
20 Great Tweets From All Time About Replace Lost Car Key
Image
The Most Important Thing!

wordpress, discuss, website, other
Image
15 Unquestionably Reasons To Love What Is The Lightest Self Propelled Wheelchair
Image
Asbestos Compensation Payouts: What Nobody Is Talking About
Image
Boiler Gas Engineer: The Good, The Bad, And The Ugly
Image
The Importance of Content Marketing
Image
10 Meetups On Accident Lawyers In My Area You Should Attend
Image
12 Companies Leading The Way In Personal Injury Compensation
Image
Tip: Leverage Indexers for Array-Like Access in C# ✨

csharp, dotnet, csharptips
Image
EchoAPI for IntelliJ IDEA: A Plugin that Boosts API Management for Java Developers

postman, java, apidocumentation, webdev
Image
How Much Do Repair Car Key Experts Make?
Image
RECOVER YOUR LOST CRYPTO WITH (FRANCISCO HACK)
Image
Frustrated with Postman's Login Requirement? Try EchoAPI Instead

postman, api, apitesting, webdev
Image
Tipe Permainan Judi Online Sportbook Yang Banyak Digemari
Image
You Are Responsible For A Asbestos Lawsuit Budget? 12 Best Ways To Spend Your Money
Image
Teknik Putuskan Permainan Judi Online Yang Pas
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .