Code coverage is the percentage of code that is covered by automated tests. As part of your CI builds, you should add a report that shows the coverage so you can track it as it goes up (of course).
Adding code coverage to your CI build is as simple as adding a new task to your build script. In other posts we’ve shown creating build scripts with various technologies. Here we’ll show adding a psake task that to invoke code coverage.
task CodeCoverage {
Write-Host("##[section]Starting: Build task 'CodeCoverage'")
copy_all_assemblies_for_test $test_dir
exec {
& $vstest_dir\vstest.console.exe $test_dir\$unitTestAssembly $test_dir\$integrationTestAssembly /TestAdapterPath:$test_dir /Logger:trx /Enablecodecoverage /Settings:$source_dir\CodeCoverage.runSettings
}
Write-Host("##[section]Finishing: Build task 'CodeCoverage'")
}
The line with vstest.console.exe
does the real work. It is also used for running tests and in this case, we’re passing in the /Enablecodecoverage /Settings:$source_dir\CodeCoverage.runSettings
parameters to get a .coverage file output. The first parameter is obvious, the second one specifies a file to customize the code coverage analysis. Typically, you don’t need one of these, but in this case we had some test assemblies to exclude as shown below. (You can also exclude with the ExcludeFromCodeCoverage
C# class attribute).
<!-- Match assembly file paths: -->
<ModulePaths>
<Include>
<ModulePath>.*\.dll$</ModulePath>
<ModulePath>.*\.exe$</ModulePath>
</Include>
<Exclude>
<ModulePath>.*CPPUnitTestFramework.*</ModulePath>
<!-- these three lines were added -->
<ModulePath>.*UnitTests.*</ModulePath>
<ModulePath>.*IntegrationTests.*</ModulePath>
<ModulePath>.*SmokeTests.*</ModulePath>
</Exclude>
</ModulePaths>
In your Azure DevOps build pipeline, add a
task after the build step, if you don’t already have one. In the
Publish Test Results
control for that task, add the .coverage file to get it copied, too.
Test results files
*.trx
***.coverage
Viewing a .coverage File
Once you build successfully, there will be a Code Coverage tab for your build. Currently all you can do is download the file.
Download the file, and open the solution that was built in Visual Studio, and open the .coverage file in Visual Studio. You should then see a Code Coverage Results
window open up.
From here you can view your coverage by assembly and drill into each one to open up the source code. The selected toggle in the tool bar above is Show Code Coverage Coloring
that puts different colors on source code depending on its coverage (if you click on it from that window, not just opening the file).
Accessing Code Coverage via REST
The Azure DevOps REST interface has a method to get the code coverage data programmatically. The flags indicate what level of detail you want to get back (1 for modules in this example). Documentation for this call is here.
$encodedPat = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(":$PAT"))
$headers = @{Authorization="Basic $encodedPat";Accept="application/json" }
$coverage = Invoke-RestMethod -Uri "https://dev.azure.com/project0795/CmBootcamp/_apis/test/codecoverage?buildId=11&flags=1&api-version=5.0-preview.1" -Headers $headers -UseBasicParsing
$coverage.value.modules | ConvertTo-Json -Depth 10
Output is something like this.
[
{
"name": "clearmeasure.bootcamp.core.dll",
"signature": "de8ee6e3-29b6-0446-8cf1-0370d457a7b2",
"signatureAge": 1,
"statistics": {
"blocksCovered": 407,
"blocksNotCovered": 146,
"linesCovered": 241,
"linesNotCovered": 86,
"linesPartiallyCovered": 11
},
"functions": [
],
"fileUrl": ""
},
{
"name": "clearmeasure.bootcamp.dataaccess.dll",
"signature": "20a6744b-6351-cd43-a8c8-44173f100580",
"signatureAge": 1,
"statistics": {
"blocksCovered": 507,
"linesCovered": 137
},
"functions": [
],
"fileUrl": ""
},
...
Including code coverage in your CI build is relatively easy, now getting to 100% is another story.
References
- Using Code Coverage
- Code Coverage runsettings file
- Improving Logs in Visual Studio Team Services (VSTS) with special ‘tags’
Originally published December 5, 2018. Information refreshed August 10, 2022.