Introduction
I had been searching for a usable and developer-friendly approach or framework to write correct, clean and readable integration tests – but I didn’t find any. So I decided to take matters into my own hands and write my own library called FScenario. This library is my vision on how tests, and more specifically integration tests, should look like.
In my opinion tests should be more readable than the production code. Why? Because the tests are the first place where new developers look to see how an API should be used, because tests are the first place to look if a defect is found, because the tests are the backbone of your entire application.
If there isn’t a test that proves your functionality, then that functionality doesn’t exist in your application!
Open-Minded Assertions
Let’s start with something that you almost always do when writing integration tests; one of the first problems you see when running your tests, especially on multiple environments, is that if you don’t take your external resources into account your assertions won’t always pass.
Maybe you have a very slow build server, or a bad network connection, or a service that locks a file, all those things can happen during your test and all those scenarios should be taken into account when you assert on anything.
What I like to call Open-Minded Assertions are assertions that take those problems into consideration. For example by polling several times for a file before allowing the test fail. Because of this, we have more reliable tests and tests that can run on multiple environments with different contexts.
Polling Targets
The library provides several ways of creating a so-called “polling function”, both usable in F# and C#: Computation Expression, F# Pipable Functions, C# Builder Interface, and predefined functions that are already configured.
The same values are always available for configuration:
- Target : where should the polling happen? (ex. file path, http endpoint, …)
- Filter : when is the result we are polling correct? (ex. 3 files in a directory, until http endpoint respond with OK, …)
- Interval : at what interval should the polling happen? (ex. every 1 second, 5 minutes, …)
- Timeout : when should the polling function time-out/stop polling? (ex. for 30 seconds, 10 minutes, …)
- Error : with what error message should the polling function let the test fail? (ex. “Polling at file: ‘temp/file.txt’ failed”, “Polling at ‘http:localhost:9090’ didn’t result in OK”, …)
See the full API reference for more information.
Zero-Waste Environment
Undoable IO
cleanUndo which does exactly that. How does the “revert” take place? Well, the cleanUndo returns a IDisposable and when this instance gets disposed the directory will get its files back. Resulting in the original situation.Dir.ensureUndo: ensures that a directory is created but reverts the ensurement when the returned disposable gets disposed.Dir.replaceUndo: replaces a directory with another but reverts the replacement when the returned disposable gets disposed.Item.replaceUndo: replaces a file with another but reverts the replacement when the returned disposable gets disposed.
Because the functions return a IDisposable instance, you control when the fixture should be reverted. This could be after every test or after the entire test suite. See the full API reference for more information about the library itself and have a look at the tests in the library if you want to see how these can be used.
No-Stress Defect Localization
I also want to stress is the importance of a No-Stress Defect Localization approach. Like I said in the beginning of this post, tests are the most important part of your applications in order to find defects (but it is not your debugger); we should make sure that the developer will be able to quickly run a test and look at the logs instead of attaching a debugger.
How we can do this? Well, every step you’re doing in your test has to be logged. This way you can go through the logs with confidence that all the actions that has taken place are there. Also, try to come up with good log messages!
Http Logging
While sending HTTP requests with the library, the request and response gets logged. You may think that you don’t need such logging but those entries could save you a lot of time the next time you’re searching for the reason something fails.
The library adds logging to the exposed functionality with the extensible Microsoft.Extensions.Logging approach. By exposing the logging factory itself, you can easily add your own logging provider. See the full API reference for more information.
Conclusion
When it comes to writing tests, developers are not always eager to code these. It could be because writing them takes a lot of time, or they don’t find them usable because they “sometimes fails” or maybe they never have actually seen a fully correct test suite with Open-Minded Assertions in a Zero-Waste Environment using a No-Stress Defect Localization.
I really encourage these developers to give FScenario, and similar libraries, a try.
Thank you so much for reading!
Subscribe to our RSS feed