Code coverage for C++

Ever since I wrote automated tests, I wondered how complete the coverage was. Of course you have a feeling which parts are better covered than others. For some legacy code you might prefer not to know at all. But I thought test coverage was something easy to do with a language running on a VM such as Java, but hard with C++. Some things are not as hard as you think, once you give it a try.

The thing that triggered my interest was the coveralls badge on the readme page of vexcl. By following it through, I learned that coveralls is just for presenting the results that are generated by gcov. Some more research showed what compiler- and linker flags I need to use. In addition I found out that lcov’s genhtml can generate nice human readable html reports, while gcovr writes machine readable xml reports. So the following is really all that needs to be added to your CMakeLists.txt:

OPTION(CODE_COVERAGE       "Generate code coverage reports using gcov" OFF)

IF(CODE_COVERAGE)
    SET(CMAKE_C_FLAGS          "${CMAKE_C_FLAGS}
        -fprofile-arcs -ftest-coverage")
    SET(CMAKE_CXX_FLAGS        "${CMAKE_CXX_FLAGS}
        -fprofile-arcs -ftest-coverage")
    SET(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}
        -fprofile-arcs -ftest-coverage")

    FILE(WRITE ${PROJECT_BINARY_DIR}/coverage.sh "#! /bin/sh"n)
    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh "lcov --zerocounters
        --directory . --base-directory ${MyApp_MAIN_DIR}"n)
    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh "lcov --capture --initial
        --directory . --base-directory ${MyApp_MAIN_DIR} --no-external
        --output-file MyAppCoverage"n)
    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh "make test"n)
    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh "lcov --no-checksum
        --directory . --base-directory ${MyApp_MAIN_DIR} --no-external
        --capture --output-file MyAppCoverage.info"n)
    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh "lcov
        --remove MyAppCoverage.info '*/UnitTests/*' '*/modassert/*'
        -o MyAppCoverage_filtered.info"n)
    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh
        "genhtml MyAppCoverage_filtered.info"n)

    FILE(APPEND ${PROJECT_BINARY_DIR}/coverage.sh
        "gcovr -o coverage_summary.xml -r ${MyApp_MAIN_DIR} -e '/usr.*'
         -e '.*/UnitTests/.*' -e '.*/modassert/.*' -x --xml-pretty"n)

    ADD_CUSTOM_TARGET(CODE_COVERAGE bash ${PROJECT_BINARY_DIR}/coverage.sh
                        WORKING_DIRECTORY ${PROJECT_BINARY_DIR}
                        COMMENT "run the unit tests with code coverage and produce an index.html report"
                        SOURCES  ${PROJECT_BINARY_DIR}/coverage.sh)
    SET_TARGET_PROPERTIES(CODE_COVERAGE PROPERTIES
        FOLDER "Testing"
    )

ENDIF(CODE_COVERAGE)

The resulting html page is very detailed and shows you the untested lines in your source files in red.
From the produced xml file it’s easy to extract the overall percentage for example. You could use this figure to fail your nightly builds when it’s decreasing.