The term I’ll use for the kind of integration tests in this post is ‘self-contained’ integration tests. These tests are written in their own ‘sandbox’, their own ‘environment’, where everything can be stubbed out or altered to verify a certain scenario. It’s very important in a team to have ‘self-contained’ integration tests, as they are independent of other tests and test suites. They have the independence advantage of unit tests and the code scope of integration tests. Let’s have a look at what practices and strategies we can practically apply in your project.
Absolutely Everything, Set it Up
The arrange-phase or setup-phase of the test will be to build up your environment. Unit tests built up objects (C#) or functions (F#) or some other small unit but here we’ll build up the entire environment that your test needs. And I mean everything. If your test needs a local folder, an environment variable, a Windows service installed and/or running… you have to make sure that the arrange-phase takes care of this.
Usually, I like to think in terms of ‘integration units’: ‘What is considered a single unit?’ The answer to that question is usually a good place to start to represent your integration unit. A practical example is: in Arcus, we have a GitHub repository that provides .NET Core project templates. In order to test these templates, we needed to make sure that our tests had access to the ‘result’ of those templates. ‘What happens when a project option is applied?’, ‘Is that file included in the end-result project?’ In this case, a project template is considered an ‘integration unit’. What do we need in the arrange phase?
- We need to install the .NET Core project template on the local system.
- We need to run the .NET Core project template with/without arguments and/or projects options.
- We need a local folder to place the result of the template.
- We need to start the end-result project.
- We need a service to access the files/available endpoints of the end-result project.
In practice, this was all collected in a single C# class where you had a ‘start new project’ functionality that did all this behind the scenes. The files/available services that you’ll probably need in the assert-phase of your test are in this case exposed via properties on this class. This ‘whole abstraction’ pattern hides the low-level parts and exposes the high-level parts.
Everything You Set Up, You Have to Tear Down
The teardown-phase of the test will be the reverse of the arrange-phase. Every setting, folder and variable you changed, you have to put back. A service that’s running, a folder that’s created. For every action, you have to come up with the reverse logic. This part is very important because the result of this phase will determine if the next test or the second test run will be on a ‘clean’ environment. There is a strong relationship between the setup and teardown-phase.
In code, we can define this as a disposable type. Creating an instance of the disposable type corresponds with the setup phase, disposing of the instance with the teardown phase. This way, both phases are hard-linked together and can never be forgotten. This pattern is something I do all the time, and here’s the breakdown of this structure in a sample code. Imagine you have run a separate worker process for your integration tests to work. This worker process needs to be started/stopped and has a need for a directory to work with.
In its simplest form, it would go like this. Imagine the ‘worker’ as our integration unit that has to be set up in our test:
And the sample implementation:
As you may notice, there’s a problem with this. It’s very easily broken and leaves your system in a corrupt state. When the directory fails to be deleted, the worker process keeps on running. That’s why you should always keep in mind failure and dispose of multiple actions separately. There’s also an argument to add some retries to these phases as we’re working with processes not always under our control.
Let’s see how this can be improved. The next sample shows how multiple actions can be performed independently from each other. The code architect in yourself may see a pattern here. Possibly, in a larger project, you could move these actions to a composable disposable type to use across ‘worker’-related types, because this pattern is useful in any context.
Local Isolated Development and Debugging
One of the main advantages of these kinds of self-contained integration tests, is independent/isolated development and testing. With that comes sometimes debugging. Definitely, when an integration test fails, you want a way to quickly spot problems. Just like in your application code, make sure you pass along an ILogger instance so that you can trace and log critical points. In the previous sample code, this would be for each setup/teardown action.
Another practice I use when writing these tests, is making sure you can configure the teardown-process. Imagine you’re experiencing difficulties with the worker process and you want to interact with it yourself to investigate the problem. When running the test, the worker process would already be closed. You could place a breakpoint somewhere, but often that’s not enough or would block other functionality. This is where this practice comes in handy.
With this configurable enumeration, you can yourself set your options to manipulate the setup/teardown. Just like you would invest in your application code, think about investigating in your testing infrastructure.
Divide and Conquer with Extensions
The trick now is to fully benefit from such a practice for your self-contained integration tests. This ‘worker’ example can have many services as properties to interact with the worker during the test run. This way you can assert things.
For example, you may want to retrieve files from that worker directory during the test run. But, when the services and properties grow too large, you may want to move them to extensions. That way, even over test suites, you can still have a clean ‘worker’ integration unit.
Conclusion
In this post we looked at some practices to write self-contained integration tests. These kind of tests are very important for a comfortable and secure testing strategy. Of course, you’ll have integration tests that will run only on a specific environment, but besides those, make sure that you have some self-contained integration tests to make your developer life more easy.
It wouldn’t be a blog post of mine if I weren’t to mention some F# code, so here it is: in my personal FScenario package, I abstracted the disposable problem described here and have created an Fixture type which is similar to a ‘worker’ here.
Happy testing, and thanks for reading!
Stijn
Subscribe to our RSS feed