all Technical posts

Reusable Integration Test Building Blocks with F# FScenario and C# Interop

Wouldn't it be great if you could make your integration tests as readable, reliable and simple as any other unit tests? This post will introduce the library FScenario, which helps you write integration tests in a more safe and correct way. Above all, it brings back the fun while coding integration tests.

Introduction

I had been searching for a usable and developer-friendly approach or framework to write correct, clean and readable integration tests – but I didn’t find any. So I decided to take matters into my own hands and write my own library called FScenario. This library is my vision on how tests, and more specifically integration tests, should look like.

In my opinion tests should be more readable than the production code. Why? Because the tests are the first place where new developers look to see how an API should be used, because tests are the first place to look if a defect is found, because the tests are the backbone of your entire application.

If there isn’t a test that proves your functionality, then that functionality doesn’t exist in your application!

Open-Minded Assertions

Let’s start with something that you almost always do when writing integration tests; one of the first problems you see when running your tests, especially on multiple environments, is that if you don’t take your external resources into account your assertions won’t always pass.

Maybe you have a very slow build server, or a bad network connection, or a service that locks a file, all those things can happen during your test and all those scenarios should be taken into account when you assert on anything.

What I like to call Open-Minded Assertions are assertions that take those problems into consideration. For example by polling several times for a file before allowing the test fail. Because of this, we have more reliable tests and tests that can run on multiple environments with different contexts.

Polling Targets

The library provides several ways of creating a so-called “polling function”, both usable in F# and C#: Computation Expression, F# Pipable Functions, C# Builder Interface, and predefined functions that are already configured.

The same values are always available for configuration:

  • Target : where should the polling happen? (ex. file path, http endpoint, …)
  • Filter : when is the result we are polling correct? (ex. 3 files in a directory, until http endpoint respond with OK, …)
  • Interval : at what interval should the polling happen? (ex. every 1 second, 5 minutes, …)
  • Timeout : when should the polling function time-out/stop polling? (ex. for 30 seconds, 10 minutes, …)
  • Error : with what error message should the polling function let the test fail? (ex. “Polling at file: ‘temp/file.txt’ failed”, “Polling at ‘http:localhost:9090’ didn’t result in OK”, …)

See the full API reference for more information.

Zero-Waste Environment

One of the issues you come across when writing integration tests, is that you have a much greater responsibility when it comes to “cleaning”. Because we deal with external sources in integration tests, there could always be some “left-overs” that weren’t cleaned-up correctly. These “left-overs” could be the cause of a test failure the next time the tests are run!
In what I like to call a Zero-Waste Environment, test fixtures or any other instances created during the test run, that are correctly disposed/removed/deleted/… afterwards. The library FScenario comes with a set of Undoable Operations that helps you with this. Let me give you an example:

Undoable IO

During the start of the test you want to “clean” a folder, meaning all the files in the folder should be removed. But after the test is complete, you actually want those files back in the folder. FScenario has a function which is called cleanUndo which does exactly that. How does the “revert” take place? Well, the cleanUndo returns a IDisposable and when this instance gets disposed the directory will get its files back. Resulting in the original situation.
There exist many more functions in the library:
  • Dir.ensureUndo : ensures that a directory is created but reverts the ensurement when the returned disposable gets disposed.
  • Dir.replaceUndo : replaces a directory with another but reverts the replacement when the returned disposable gets disposed.
  • Item.replaceUndo : replaces a file with another but reverts the replacement when the returned disposable gets disposed.

Because the functions return a IDisposable instance, you control when the fixture should be reverted. This could be after every test or after the entire test suite. See the full API reference for more information about the library itself and have a look at the tests in the library if you want to see how these can be used.

No-Stress Defect Localization

I also want to stress is the importance of a No-Stress Defect Localization  approach. Like I said in the beginning of this post, tests are the most important part of your applications in order to find defects (but it is not your debugger); we should make sure that the developer will be able to quickly run a test and look at the logs instead of attaching a debugger.

How we can do this? Well, every step you’re doing in your test has to be logged. This way you can go through the logs with confidence that all the actions that has taken place are there. Also, try to come up with good log messages!

Http Logging

While sending HTTP requests with the library, the request and response gets logged. You may think that you don’t need such logging but those entries could save you a lot of time the next time you’re searching for the reason something fails.

The library adds logging to the exposed functionality with the extensible Microsoft.Extensions.Logging approach. By exposing the logging factory itself, you can easily add your own logging provider. See the full API reference for more information.

Conclusion

When it comes to writing tests, developers are not always eager to code these. It could be because writing them takes a lot of time, or they don’t find them usable because they “sometimes fails” or maybe they never have actually seen a fully correct test suite with Open-Minded Assertions in a Zero-Waste Environment using a No-Stress Defect Localization.

I really encourage these developers to give FScenario, and similar libraries, a try.

Thank you so much for reading!

Subscribe to our RSS feed

Let's talk, what's your first name?

Press Enter

Press Enter

Hi , what's on your mind?

Press Enter

Great! Leave your email and we'll be in touch
Submit
Thanks, we’ll be in touch soon!