NEW! Webinar: Create Clarity – Don’t Run Your Project Blind

Running unit and component-level integration tests in CI builds

Staff Writer
November 7, 2018
Azure DevOps

Share This Article:

Running automated tests in continuous integration (CI) builds are important to ensure that robust applications are deployed to production. Different types of tests are run at each phase of the deployment cycle (Dev, UAT, Staging, etc.). Unit tests are run on every build. Then up through production Integration and Acceptance tests are run.

The CI build runs as many automated tests as possible in the industry-recommended ten minutes. Frequently, all of the unit tests and component-level integration tests can be run within that time.

Testing Frameworks

All the types of test described below will be written with some type of testing framework. There are many of them and they all work in similar fashion, and can be run by the Visual Studio Task in Azure DevOps pipelines. xUnit.net is a free, open source, community-focused unit testing tool for the .NET Framework. Written by the original inventor of NUnit v2, xUnit.net is the latest technology for unit testing C#, F#, VB.NET and other .NET languages. xUnit.net works with ReSharper, CodeRush, TestDriven.NET and Xamarin.

For any given aspect of the code, its tests are usually encapsulated in a C# class and that class should test only that aspect. A test class can have setup and teardown methods that prepare the test and clean up after it, if necessary. It will have several other methods that test that the code works correctly, by asserting that something is true.

Below is a simple xUnit tests that verifies that an Add function is returning a correct result. The class should have many methods that ideally cover all code paths, negative testing, and edge cases. All the test frameworks have a zoo of methods for asserting things like Equal, NotEqual, Null, Throws, etc.

    [Fact]
    public void PassingTest()
    {
        Assert.Equal(4, Add(2, 2));
    }

To run tests in a Azure DevOps pipeline, you can invoke the test running in a script, or use the Visual Studio Test task. That task uses adapters to run tests from many different frameworks. See the documentation for details.

Unit Tests (UT)

Unit tests test code at the lowest level, or unit of code. These tests can be run in isolation without any other supporting processes and should not change any state. If the tested unit needs an external process it should be simulated, or mocked, to supply the tested code with simulated output, both good and bad.

Unit tests should be added to each solution or group of interrelated projects. Although it is best to strive for 100% code coverage, initial unit tests should target covering the most critical aspects of a given project. After the creation of the unit test projects, all future developed features should be covered under respective, new units.

By writing tests with full coverage, the developer can be confident that any changes made to the code didnâ€™t break anything.

Integration Tests (IT)

Integration tests determine if independently-developed units of software work correctly when they are connected to each other. ITs will run tests against systems such as a database, or REST service to verify connectivity and that the service returns the correct data.

Integration tests run in a deployed environment, so are executed in the Release Pipeline. The source for the tests should be built (if necessary) during the Build and saved as an artifact for the Release to run to ensure the tests match the deployed code.

Writing and running Integration Tests are the same as for UTs, using one of the testing framworks. Since these tests cross system boundaries, they usually have special code for setting up the test and tearing it down. Creating an environment with valid sample data, and being able to restore it for each test can be a big effort.

Acceptance (Automated UI) Tests (AT)

Like IT, typically you will run acceptance tests in the release workflow after the application is deployed (usually to a QA environment). Initially, the developer or tester simply clicks through the UI controls to verify that things are working correctly. However, it can be beneficial to include testing of the various UI controls in the application. Automated UI tests can verify that code changes do not impact the functionality of UI controls of the application.

Tests can be written with different frameworks. Depending on the application, one may be easier than another. Clear Measure recommends using Selenium.

Like the ITs, these tests require an environment with consistent test data each time they are run since often a UI test depends on specific data in the system to consistently work.

Summary

With the rapid pace of continuous integration and deployment, a system that does not include automated tested will bound to ship with bugs. Striving for 100% is a good goal, and although difficult to reach, is a good target.