wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Monday, July 24, 2017 8:24 AM

Stijn Moreels by Stijn Moreels

In this part of the Test Infected series, I will talk about the ignorance of tests and how we can achieve more ignorance.
The short answer: if it doesn’t contribute to the test, hide/remove it!
The reason I wrote this post is because I see many tests with an overload of information embedded, and with some more background information people may increase the ignorance of their tests.

Introduction

In this part of the Test Infected series, I will talk about the ignorance of tests and how we can achieve more ignorance.

The short answer: if it doesn’t contribute to the test, hide/remove it!

The reason I wrote this post is because I see many tests with an overload of information embedded, and with some more background information people may increase the ignorance of their tests.

Ignorance

Test-Driven Discovery

Sometimes people write tests just because they are obliged to do so. Only when someone is looking over their shoulder they write tests, in any other circumstances they don’t. It’s also crazy to see people abandon their practices (TDD, Merciless Refactoring, …) the moment there is a crisis or a need for a quick change or something else that causes stress to people.

The next time you’re in such a situation, look at yourself and evaluate how you react. If you don’t stick with your practices in those situations, then do you really trust your practices at all?If you don’t use your practices in your stress situations, abandon them, because they wouldn’t work for you (yet).
This could be a learning moment for the next time.

Now, that was a long intro to come to this point: what after we have written our tests (in a Test-First/Last approach)?

Test Maintenance

In my opinion, the one thing Test Ignorance is about, is Test Maintenance. When there’s some changes to the SUT (System Under Test), how much do you have to change of the production code and how much of the test code?

When you (over)use Mock Objects (and Test Doubles in general), you can get in situations that Gerard Meszaros calls Overspecified Software. The tight-coupling between the tests and the production code is causing this Smell.

But that’s not actually the topic I want to talk about (at least not directly). What I do want to talk about are all those tests with so much information in them that every method/class/… is Obscured.

People read books about patterns, principles, practices… and try to apply them to their Production Code, but forget their Test Code.

Test Code should be as clear than the Production Code.

If your method in production has 20 lines of code and people always lose time to read and reread it… (how many times do you reread a method before you refactor?), you refactor it to smaller parts to improve usability, readability, intent…

You do this practice in your production code; so, why wouldn’t you do this in your test code?

I believe one of the reasons people sometimes abandon their tests, is because people think they get paid for production code (and not because of their lack of discipline). It’s as simple as that. But remember that you get paid for stable, maintainable, high quality software and you can’t simply deliver that without tests that are easily maintainable.

"Ignorance is Bliss" Patterns

“I know this steak doesn't exist. I know that when I put it in my mouth, the Matrix is telling my brain that it is juicy and delicious. After nine years, you know what I realize? Ignorance is bliss.”

- Cypher (The Matrix)

Now, when you understand that your test code is just as much important than your production code, we can start by defining our Ignorance in our tests.

There are several Test Patterns in literature that support this Test Ignorance, so I’ll give you just the concepts and some quick examples.

This section is about readability and how we can improve this.

Unneccessary Fixture

The one place where you could start and where it is most obvious there’s a Test Smell, is the Fixture Setup. Not only can this section be enormous (I’ve seen gigantic fixtures) and hard to grasp, there’re also hard to change and so, to maintain.

Look at the following example. We need to setup a “valid” customer before we can insert it into the Repository. In this test, do I really need to know what all the different items are to make an invalid customer. Do we need all of them? Maybe it’s just the id that’s missing, but that could be autogenerated, or maybe the address doesn’t exist, …

Only show what I need to make the test pass.

We can change the example with a Parameterized Creation Method as an example of the One Bad Attribute Test Pattern. In the future, we could also parameterize the other properties if we want to test some functionality that depends on this information. If this isn’t the case, we can leave these initializations inside the Creation Method for the customer instead of polluting the test with this unnecessary information.

Now, if we want to act as a fully Test Infected person, we can also Test-Drive these Creation Methods. Next time you analyze the code coverage of your code, include the test projects and also make tests for these methods! This will increase your Defect Localization. If there's a problem with your fixture and not the test that uses this fixture, you will see this in your failed tests and know that you have a problem with the Fixture and not the test itself.

Also note that this newly created method is only accessible within this test class. If we want to write tests with the same Fixture, we can extract this logic in its own class.

In either way, we have made our intentions clear to the test reader. I always try to ask the following question to the test: “Do you really care if you know this?”.

Again, the test can be made clearer if we send the argument that makes the customer invalid with it, so we know what’s the cause why the customer isn’t inserted. If we move the “id” somewhere else, we won’t know what causes it and would made the test more Obscure.

I see some reasons why a Test Fixture can be big:

  • The Fixture has a lot of “setup-code” in place because the SUT is doing too much. Because the SUT is doing all these steps, we must build our Fixture with a lot of info and behavior. Otherwise, the SUT will see the Fixture as invalid.
  • The Fixture is the smallest possible for exercising the SUT and the SUT is a Complete Abstraction, but needs nonetheless a Fixture that needs some lines to setup before the Fixture is valid.
  • The Fixture contains some unnecessary information that doesn’t contribute the result of the test but is embedded in the test anyway.

So, there are a lot of different possibility why a Fixture can be big and the solution is for all these situations the same: make the Fixture as small as possible + only add the information to the test, relevant to the result of the test . Contribute or get out.

Now, if you move ALL the Fixture code somewhere else (so extracting too much), you also have a problem. Test readers will now see some Magic Fixtures in place that act as Mystery Guests which can result in Fragile Tests.

Obscured by Eagerness

Sometimes, I encounter tests that are “Obscured by Eagerness”. A Test can be “obscure” for different reasons. One can be because we want to assert too much in a single test, another can be because we want to “set-up” too much in a single run, and yet another can be because we combine tests in a single test run by exercising multiple actions on the SUT.

To summarize:

  • Eager Assertion: assert on too many state and/or behavior in a single run.
  • Eager Fixture: set up unnecessary fixture (see previous section).
  • Eager Exercises: exercise multiple actions on the SUT to combine tests.

I’ve seen people defend tests with more than 20 assert statements because they still tested a “single unit” outcome. Sometimes you have functionality that looks like you have to write 20 assert statements or more, but instead of writing those statements you should ask yourself: What are you trying to test?

By explicitly asking yourself this question, you often come up with surprising results.

Because the assert-phase of the test (in a Four Phase Test) is important to verify the state of the test (failed or succeed), I always try to start by writing this phase first. It forces you to think about what you trying to test and not what you need to set up as Fixture. By writing this phase first, you’re writing your test from bottom to top and only define what you really need. This way (like writing tests for your production code), you only write what you need.

Previous snippet is a perfect example of how we can abuse the Assert-Phase. By placing so many asserts in a single spot, we obscure what we really trying to test. We need to test if the message is serialized correctly; so instead of manually getting each element, why not assert on the whole xml?

We create an expected xml string and verify if this is the same as the actual serialized xml string.

Conclusion

Writing tests should be taken as serious as the production code, only then we can have maintainable software solutions where developers are eager to run tests instead of ignoring them.

The next time you write a test, try to think firmly about the test. What should I know, what do I find important to exercise the SUT, what do I expect… this way you can determine what items are important and which aren’t.

I sometimes "pretend" to be the test case:

“Do I care how this Fixture is set up?”
“Must I know exactly how to assert all these items?”
“Have I any interest of how a ‘valid’ object looks like?”
“What do I really want to test and what information only pollutes this?”
“Do I care that these actions must be executed before the exercise of the test?”

Tests only need to know what they need to exercise the SUT, nothing more, but equally important: nothing less!

Categories: Technology
Tags: Code Quality
written by: Stijn Moreels

Posted on Wednesday, July 12, 2017 9:45 AM

Stijn Moreels by Stijn Moreels

In this part of the Test Infected series, I will talk about how code is hard to test – both in a Test-First mindset and without.

Hard-to-Test Code

By “hard”, I mean anything that is uneasy, sloppy, frustrating or annoying, … anything that makes you sigh. All that can be categorized as “hard”.

TDD or Test-Driven Development is a lot more than just writing tests first, it’s also about Designing Software. You think about a lot of things during the writing of tests, and all those things together guards you from writing Hard-to-Test Code.

When I write tests, I want an easy way to exercise and verify the SUT (System Under Test). The less code I need to write to do that, the better. The clearer the test intent, the better. The easier the test, the better.

Obscured Fixture Setup

What do I mean with a Fixture in this paragraph? Anything that you need to do so you can exercise the SUT. This could mean that you first have to initialize the SUT with some valid arguments, it could mean that you must insert some Dummy Data into a datastore, it could mean that you must call some methods of the SUT...

According to Kent Beck: anything that “Sets the Table” for the SUT.

This section is about the maintainability of the Test Fixture and how we can improve this.

Discovery

With a Complex Fixture Setup, I mean that I must write a lot of code to “set this table”. I must admit that  I quickly assign a fixture as being “complex” – but that’s a good thing I guess.

Look at the following snippet. It’s a good thing that we “spy” Client instead of actually sending a mail, but also note that your eyes are attracted to the strings in the Attachments and Header of the valid message instead of the actual testing and verifying.

I don’t like complex, big, or hard-to-understand Fixtures. I want clear visual of what is tested and how. Nothing more, nothing less. Of course, I don’t know if you find this complex, maybe you don’t (because you wrote it), I just don’t like big methods I guess.

We have 16 lines, with 3 comments, 2 blank spaces, 2 braces, 7 lines of which are Fixture Setup.

Causes

You could think of many causes to having a Complex Fixture Setup.

  • You could have a Tightly-Coupled system which forces you to create all those extra objects and initialize them with the right values.
  • Your test includes information which doesn’t really matter in the context of the test; this way introducing a Polluted Test. This could happen in a Copy-Paste programming environment in which you just copy the Fixture of another test.
  • It could also happen if there wasn’t enough research done to maybe Fake the Fixture and thereby avoiding unnecessary setup code.

Impact

Now, we have a Complex Fixture – so what?

Let’s look at the impact a Complex Fixture could have on your development. What if I want to test some unhappy paths for the Client. What if we want to test the creation of the Message with Constructor Tests. What if we want to test with a single Attachment instead of two…

Al those tests would require a similar Fixture Setup.

If you have Cut-and-Paste developers in your team, you would have a lot of Test Duplication. Which again result in a High Test Maintenance Cost.
Besides the fact of duplication, it isn’t clear to me that we are testing a “valid message”. Does it have to do with the header value? Does it have to do with the attachments? Does it have to do with something else? …

What do I have to do to create valid mail message. Does this message require attachments? By not having a clear Fixture Setup, we don’t have a clear Test Overview.

Possible Solution

The first thing you should do is to eliminate all the unnecessary information from your test. If you don’t use it/need it – don’t show it.
I only want to see what’s really important to the test to pass.

If you, after that, still have a big Fixture to set up – place it in Parameterized Creation Methods so you can only send the values to the Creation Methods that are valuable for the test. This way you resolve the duplication of the tests.

Also make sure that you don’t have any duplication in your Implicit Fixture (typically in some kind of “setup” method or constructor) for setting up a Datastore for example.

Missing Object Seam Enabling Point

What is an Object Seam? Michael C. Feathers introduced this and called it: “A place where you can alter behavior without editing that place”.

Every Seam must have an Enabling Point where the decision for one behavior or the other can be made. In Object-Oriented languages, we can use this method by introducing Test Doubles by which we implement another version of the dependency or other kind of object we need to exercise the SUT.

Discovery

Not having an Enabling Point for the SUT makes our code Hard-to-Test. This can happen in many situations – especially when you have chosen some kind of design decision and everything must make room. (This sounds a bit like a Golden Hamer Smell to me).

Please don’t look at the names, it’s only an example.

The following example shows how the Message Service doesn’t contains any enabling point. We are bound to use the file system if we want to test the Message Service. A possible solution could introduce a Fake Datastore (probably in-memory) and send it to the Message Service.

Also note that we can’t even write a valid assertion. Yes, we could check if the File Based Data Store has written something on the disk. But I hope that you can see that this isn’t the right way to assert the Message Service.

We would have to write code that assert the functionality of the File Based Data Store and not from the Message Service.

Ok, it was a very “simple” example of a SUT could miss an Enabling Point. Let’s make it a tiny bit harder.

The File Sender always uses XML serialization/deserialization if we want to write a file. We always must use the Message Helper (what kind of name is that?) if we want to write a file.

These Utility Classes are the result of thinking in a Procedural way, not in a OOP way. Because all these static classes are used in this class, we have no choice than to also test them if we want to test the File Sender. If we Unit Test this class, we aren’t testing a “unit”, we are testing 3 (!) classes.

Whenever I see the name “helper” appear, I immediately think that there is room for improvement in the design. For a start, please rename “helper” to a more appropriate name. Everything could be a helper for another class but that doesn’t mean we must name all our classes with the prefix “helper”.

Try to move those functionalities in the appropriate classes. Our Message could for example have a property called IsEmpty instead of outsourcing this functionality in a different class.

Functional Languages have the same need to inject Seams. Look at the following example of the same functionality:

If we want to change the Message Helper or the serialization, we must inject functions in our “Write to File” function. In this case, our Stub is just another function.

Again, don’t look at the names, or functionality – it’s just to make a point on the Enabling Point of our Object Seams.

Causes

You could think of many causes of a Missing Enabling Point:

  • If the Dependencies are implemented right inside the SUT – which would indicate a Tight-Coupling (again); and therefore, we cannot implement our own version of the dependency.
  • Utility Classes are a result of Procedural Thinking in a OOP environment. (Yes, maybe there are some exceptions – maybe) which result in Static Dependency with the SUT. We cannot alter these behaviors of these static classes in our test.
  • The SUT may do a lot of work inside the constructor which always need to run if we want to exercise the SUT – thereby limiting our capabilities of placing an Enabling Point.
  • Having a chain of method calls could also indicate this Tight-Coupling only in a more subtle way. If we have a single dependency but we have to call three methods deep before we have the right info, we have the same problem. It violates the "Don't Talk to Strangers" design principle.

Impact

By constantly missing an Enabling Point for your Seam; you are making a design that isn’t reusable for other purposes.
Sometimes the reason behind the absence of Enabling Points lies in the way the responsibilities are spread across the classes and not wrapped in the right classes.

Maybe I’m a bit allergic to Utility Classes.

Possible Solution

Placing an Enabling Point in our SUT. That should be our solution. We need some kind of Observation Point we can use to alter behavior, so we can verify the outcome.

Note that the observation can be direct or indirect, just like the control point (or Enabling Point) of our SUT. Use different kind of Test Doubles to achieve these goals.

We use a Test Stub to control the Indirect Inputs of the SUT; we use a Mock Object for verification of the Indirect Outputs.

Classes which have private information, behavior, … these classes can maybe expose their information or behavior in a subclass. We could create a Test Specific Subclass which we can use to exercise the SUT instead of the real one.

But be careful that you don’t override any behavior you are testing. That we lead to False Positive test cases and would introduce paths in your software that are never exercised in a test environment.

In Functional languages, everything is a function; so, we could introduce a Stub Function for our injection of data, and a Mock and/or Stub Function for our response and verification, … so we have an Enabling Point for our SUT.

Over-Specified Software by Mocking

I already said it in previous posts and sections: you must be careful about Mocking and what you try to Mock. In another post, I mentioned the Mock Object as Test Double to assert on indirect outputs of the SUT. This can be useful if you can’t verify any other outside observable behavior or state of the SUT.

By using this Mock Object we can in fact verify the change of the SUT.

Discovery

Yes, we can verify the change of the SUT; but have we also a maintainable change of the Mock Object?
If we need to change some signature of the Mock Object, we need to alter the change throughout all the Mock Objects and there direct assertions to complete the change in signature.
If we mock out everything of the SUT and thereby Tight-Couple our Test Double with the SUT, we have Over-Specified Software by Mocking too much.

Causes

Multiple situations can cause a SUT being Tight-Coupled to the DOC (Depend-On Component, in this case the Mock Object).

  • Writing Tests After implementation can cause this situation. If you have developed a Hard-to-Test SUT, you may have encounter a SUT that only can be exercised/tested (and so, verified) by injecting a Mock Object and assert on the indirect outputs of the SUT.
    Sometimes, the assert-phase of these tests aren’t the whole picture we want to test but only a fragment; a side-effect. By using this extra side-effect, we have made our test code Tight-Coupled on our SUT.
  • Testing Unnecessary Side-Effects can also cause Over-Specified Software. If we assert on things we don't necessary need in our test or things that do not add any extra certainty to our test case; we should remove those assertions. Testing on “extra” items doesn’t result in more robust software but rather in Over-Specified Software.

Impact

So, let’s say you’re in such a situation; what’s the impact in your development work?
Just like any software that is Tight-Coupled, you have the cost of maintenance. If you’re software is tested in a way that the slightest change in your SUT that doesn’t alter any behavior of the SUT result in a failed test; you could state that you have Over-Specified Software.
Any change you make is a hard one, which result that developers will make lesser changes. Lesser changes/refactorings/cleanup… will result in lesser quality of your software.

Possible Solution

People tend to forget the Dummy Object when they are writing tests. The purpose of the Dummy Object is to fulfil the SUT needs without actually doing anything for it. Passing “null” or empty strings are good examples, or objects that are empty (and maybe throw exceptions when they are called to ensure that they aren’t called during the exercise of the SUT).
Not everything needs to be a Mock Object. And just to be clear A Mock isn’t a Stub!

You’ll be amazed how many unnecessary data you write in your tests when you start removing all those Mock Objects and replace them with lighter objects like Dummy Objects.

Yes, Mock Objects are absolutely necessary for a complete developer toolset for testing; yes, sometimes Mock Objects are the only possible solution to verify indirect outcomes of the SUT; yes, sometimes we need to assert on indirect output calls directly…
But certainly, not always. Try using another Test Double first instead of directly using a Mock Object. Just like you’ll use an Inline Fixture first before moving to a Shared Fixture.

Besides the fact that you can change your Test Double, you could also look at WHAT you want to test and may come up with some refactorings in your SUT to verify the right state or behavior. The best solution to have code that is easy to test; is writing your tests first, which result immediately in testable code.

Asynchronous Code

Some small part about Asynchronous Code, because it's too much to talk about in such a small section.

The problem with async code, is that we don't always have the same context in which we can assert for the right outcome. Sometimes we use a Polling functionality to get the work done for example. This will (of course) result in Slow Tests, but sometimes we don't have control of the running process.

In the book xUnit Test Patterns we've seen that we can use a Humble Object which extracts the async code, so we can make sync calls in our test. In my previous post, I talked about a Spy and used a Wait Handle, to block the thread before we succeed the test; this can also be a solution (if it's implemented right; timeout, ...).

The xUnit framework (not the xUnit family!; .NET xUnit != xUnit Family) written for .NET, has support for async methods which makes sure that we can assert in the right task-context.

Conclusion

So many classes are already in play that are more difficult to test; that’s why my motto is to not make this pile of classes any bigger and to write in a Test-Driven way easy-to-read/easy-to-maintain code every day. Because every day can be that day where you must change a feature, add/modify/remove functionality, or anything else that include change.

Tests are there to help, not to slow you down. In fact, by writing tests you work more productively, efficiently, safer, rousted, …

So, don’t write any Hard-to-Test code but write code that grows incrementally from your tests.

Categories: Technology
Tags: Code Quality
written by: Stijn Moreels

Posted on Friday, July 7, 2017 12:00 PM

Stijn Moreels by Stijn Moreels

In this part of the Test Infected series, I will talk about the Test Doubles. These elements are defined to be used as a “stand-in” for our SUT (System under Test). These Test Doubles can be DOC (Depend on Components); but also, other elements we need to inject to exercise the SUT.

Introduction

This is probably the first post in the Test Infected series. The term “test infected” was first used by Erich Gamma and Kent Beck in their article.

“We have been amazed at how much more fun programming is and how much more aggressive we are willing to be and how much less stress we feel when we are supported by tests.”

The term "Test-Driven Development" was something I heard in my first steps of programming. But it was when reading different books about the topic that I really understood what they meant.

The Clean Coder by Robert C. Martin talks about the courage and a level of certainty of Test-Driven Development. Test-Driven Development: By Example by Kent Beck has taught me the mentality behind the practice and Gerard Meszaros with his book xUnit Test Patters has showed me the several practices that not only improved my daily development, but also my Test-First mindset. All these people have inspired me to learn more about Test-Driven Development and the Test-First Mindset. To see relationships between different visions and to combine them the way I see it; that's the purpose of my Test Infected series.

In this part of the Test Infected series, I will talk about the Test Doubles. These elements are defined to be used as a “stand-in” for our SUT (System Under Test). These Test Doubles can be DOC (Depend on Components); but also, other elements we need to inject to exercise the SUT.

I find it not only interesting to examine the theoretical concept of a Test Double, but also how we can use it in our programming.

Types

No, a Stub isn’t a Mock; no, a Dummy isn’t a Fake. There are differences in the way we test our code. Some test direct inputs other indirect outputs. Each type has a clear boundary and reason to use.

But be careful, overuse of these Test Doubles leads to Over Specified Software in which the test is Tightly-Coupled to the Fixture Setup of the tests which result in more refactoring work for your tests (sometimes more than the production code itself).

Test Code must be as clear, simple and maintainable… as Production Code – maybe even more.

Dummy Object

We use a Dummy Object if we want to inject some information that will never be used. null (C#), None (Python) … are good examples; but even “ignored data” (string) are valid Dummy Objects. If we’re talking about actual valid objects, we could throw exceptions when the methods of that object are called. This way we make sure that the object isn’t used.

We introduce these kinds of objects because the signature of the object to test requires some information. But if this information is not of interest of the test, we could introduce a Dummy Object to only show the test reader the related test information.

We must introduce custom Dummy Objects if the SUT doesn’t allow us to send null / None

Test Stub

In the literature, I found two different types of Test Stubs. One that returns or exposes some data, which can be used to validate the actual outcome of the System under Test (SUT); this is called a Responder and one that throws exceptions when the SUT interacts with the Stub (by calling methods, data…) so the Unhappy Path is being tested; this is called a Saboteur.

But, I encountered a possible third type which I sometimes use in test cases. I like to call it a Sink but it’s actually just a Null Object. This type of Stub object would just act as a “sink”, which means that the Stub isn’t doing anything with the given data. You could use a Sink in situations where you must for example inject a “valid” object but don’t feel like the test case doesn’t really care about what’s happening outside the SUT (in what cases does it?).

By introducing such an Anonymous Object, you let the test reader know that the object you send to the SUT is not of any value for the test.

This kind of “stubbing” can also be accomplished by introducing a new Subclass with Test Specifics and override with empty, expected or invalid implementations to test all paths of the SUT.

Following example shows how the Basket gets calculated with a valid (Anonymous) and invalid (Saboteur) product. I like to call valid items “filled” or “anonymous” to reference the fact that I don’t care what it contains or is structured. You can use “saboteur” to indicate the you have a “invalid” product in place that throws exceptions when it gets called.

I don’t know why, but sometimes, especially in this case where you have a several valid and a single invalid item – the setup reminds me of a pattern called Poison Pill. This is used in situations where you want to stop an execution task from running by placing a “poison pill” in the flow.

This type of Stub isn’t a Dummy object because the methods, properties, etc… are called. Also note that there’s a difference between a Saboteur and a Dummy Object which throws exceptions when called. The Saboteur is used to test all the paths of the SUT; whether the Dummy Object guards against calls that aren’t supposed to happen (which result in a test failure).

You would be amazed what a Stub can do in your design. Some developers even use this stub later as the actual production implementation. This is for me the ultimate example of Incremental Design. You start by writing your tests and incrementally start writing classes that are dependencies of your SUT. These classes will eventually evolve in actual production code.

Now, here is an example of a Stub. The beauty of Functional Programming, is that we can use Object Expressions. This means we can inline our Stub in our test.

Java also has a feature to define inline classes and override only a part of the methods you exercise during the test run

 

  • To decrease the Test Duplication, we can define Pseudo Objects for our Stubs. This means we define a default implementations that throws exceptions for any called member (like a Dummy for example). This allows us to override only those members we are interested in, in our Stub.
  • During my first experience with Kent Beck's Test-Driven Development, I came across the Self-Shunt idea. This can actually be any Test Double, but I use it most of the time as a Stub. Here, we use the Test Class itself as Test Double. Because we don't create an extra class, and we specify the return value explicitly; we have a very clear Code Intent. Note that I only use this practice if the Test Double, can't be reused somewhere else. Sometimes your Test Double start as a Self-Shunt but can grow to a full-blown Stub.

Test Spy

Ok, Stubs are cool – very cool. The Saboteur is especially useful to test the unhappy/rainy paths throughout the SUT. But there’s a downside to a pure stub: we cannot test the Indirect Output of our SUT.

That’s where the Test Spy comes in. With this Test Double, we can capture the output calls (the Indirect Output) of our SUT for later verification in our test.

Most of the time, this is interesting if the SUT doesn’t return anything useful that we can use to verify if the test was successful. We can just write the ACT statement and no ASSERT statement and the test will automatically result in a test failure if any exception during the exercise of the SUT is being thrown.

But, it’s not a very explicit assertion AND (more importantly), if there are any changes to the SUT, we cannot fully verify if that change in behavior doesn’t break our software.

When developing a logging framework (for example); you will have a lot of these situations because a log-function wouldn’t return anything (the log framework I came across didn’t). So, if we only get a void (in C#), how can we verify if our log message is written correctly?

When working in an asynchronous environment; Test Spies also can be useful. Testing asynchronous code will always have some blocking system in place if we want to test Indirect Outputs – so a Test Spy is the ideal solution.

By hiding the blocking mechanism, we have a clear test and the test reader knows exactly what the purpose of the test is, what the SUT should do to make the test pass, and what the DOC (Depend-on Component) do in the background to make the right verification in our assert-phase.

All of this makes sure that we have a true test positive.

The time-out is (off course) context specific – try to limit to the very minimum; 5 seconds is a very long time for a single unit test to pass but not for an integration test.

Mock Object

If we want to test Indirect Outputs right away and not at the end of the test run (like a Test Spy uses “is called” in the assert-phase); we can use a Mock Object.

There’s a subtle difference with a Mock Object and a Test Spy. A Spy will capture its observations so that it can be verified in a later part (in the assert-phase); while a Mock will make the test fail when it encounters something that was not expected.

Of course, combinations can be made, but there’s something that I would like to warn you about the Mock Object. Actually, two somethings.

1) One must be careful what he/she mocks and what he/she exercises in the SUT. If we mock too much or mock the wrong parts, how do we verify that our SUT will survive in the “real world” where there aren’t any Mock Objects that return just the data the SUT expect?

2) One must be careful that it doesn’t use the Mock Object for all his/her tests. It will result in a Tight-Coupling between the test cases and the SUT. Especially when one uses mocking frameworks, it can be overused. Try to imagine that there must change something in your SUT. Try to go down the path how many Mock Objects you must change in order to get that change in your SUT.

Tests are there to help us, not to frustrate us.

These two items can also be applied on Test Stubs for example, if we specify too much information in our Indirect Input. The  different with a Mock is that we validate also the Indirect Output immediately the output and not in our Assert-phase. Tight-Coupling and overusing any pattern is a bad practice in my opinion. So, always start with the smallest: can you use a Dummy? Then a Stub? Maybe we can just Spy that? Ok, now we can use a Mock.

Look at the following example: we have a function that transforms a given input to an output, but only after we asserted on the expected input. This is a good example of how we assert directly and giving expected output for our SUT.

Fake Object

The last Test Double I would like to discuss, the Fake Object. This Test Double doesn’t always need be configured. An object that is a “fake” is actually a full-implementation object that implement the functionality in such a way that the test can use it during the test run.

A perfect example is the in-memory datastore. We implement the whole datastore operations, all within memory so we don’t need a full configured datastore in our tests.

Yes, of course you must test the datastore connection but with a Fake Object in place you can limit the tests that connect to the “real” database to a minimum and run all the other tests with a “fake”

Your first reaction for external components should be to check if you can fake the whole external connection. Tests that use in-memory storage rather than the file system, datastore, network connectivity… will run a lot faster – and therefore will be run a lot more by the developers.

This type of Test Double is different from the others, in a way that there is no verification in place. This type of object “just” replaces the whole implementation the SUT is dependent on.

Conclusion

Honestly, I think that the reason why I wrote this blog post is because I heard people always talk about “mocks” instead of the right words. Like Martin Fowler says in in his blog post: “Mocks aren’t Stubs”.

I know that in different environments people use different terms. A Pragmatic Programmer will use other words or the same for some Test Doubles than someone from the Extreme Programming background. But it is better that you state what you mean with the right terminology, than to call everything a “mock”.

What I also wanted to show was, that a Test Double isn’t “bound” to Object Oriented Programming or Functional Programming. A Test Double is a concept (invented by Gerard Meszaros) that alter the behavior of your SUT in such a way that the test can verify the expected outcome.

It’s a concept, and concepts can be used everywhere.

Categories: Technology
written by: Stijn Moreels

Posted on Friday, June 23, 2017 10:39 AM

Stijn Moreels by Stijn Moreels

As you may know, from previous blog posts, I use FAKE as build script tool to automate my compiling, testing, inspection, documentation and many other things. FAKE has already a lot of functionality in place, but they didn’t have any support for StyleCop. Until now.

StyleCop and F#

Why StyleCop?

StyleCop is a tool that analyzes your source files with the default Microsoft coding conventions describing how code should look like.

It’s not that I agree with every StyleCop rule, but there are more rules that I do agree on then otherwise.

FAKE already had support for FxCop, but since ReSharper had Intellisense support for StyleCop in place, I found it reasonable that my automated build (local and remote) is depending on an analyzing tool that I both can use in my development practice as in my automated build.

StyleCop Command Line

FAKE is written in F#, which means the whole .NET framework is available to us. Besides working with FAKE and some personal fun-projects, I didn’t have much experience with F# so it was a fun challenge.

StyleCop already has some command line tools that you can use to analyze your source files, so in theory my StyleCop implementation of F# could just use some of those tools and pass the right arguments with it?

F# Implementation

F# is a functional language with imperative features (such as mutable values, foreach…). My goal was to write a purely functional implementation of the command line tools that analyzed the source files.

I’m not going to run through every bit of the code, you can do that yourself with the link on the end of this post.

Before we get started, some practical numbers:

  • I wrote the implementation in 2-3 hours (so it certainly can be improved)
  • The single file contains around 100 lines of code

Just for fun, I checked the length of some C# implementation, and found that all together they had 650 lines of code.

Imperative to Functional

For Each For Each

One of the things I had to implement, was the possibility to analyze Source, Project and Solution Files. Source files could be analyzed directly; project files must first be decomposed in source files and solution files must first be decomposed in project files.

When I looked at the C# implementation, you could see that they had implemented a foreach, in a foreach, in a foreach, to get the three different levels of lists.
Each source file must be wrapped in a StyleCop Project object so it can be analyzed, so you must indeed go through every project file and solution file to obtain all those underlying source files.

Functional programming has different approaches. What we want to do is: “create a StyleCop Project for every source file”. That was my initial idea. I don’t want to know where those files came from (project, solution). I came up with this solution:

Every StyleCop Project must have an identifier which must be incremental by each source file it analyzes.

In my whole file, there are no foreach loops, but this function is called three times directly and time indirectly. So, you could say it’s the core of the file.

The function takes a start ID, together with the files to run through, to create StyleCop Project instances. It’s a nice example of the Tail Recursion in Functional Programming, where you let a function run through every element in a list (via the tail).
At the end, we combine all the projects to StyleCop Project instances and return that list.

Shorthanded Increment

In the C# implementation, they used the shorthanded increment (++) to assign the next integer as the Project Id. In the previous snippet, you see that the ID is also sent with the function and is increased internally. This way we can reuse this function because we could start from zero, but from any valid integer number. And that’s what I’ve done.

The source files can call this function directly, project files go through the source files and the solution files go through the project files but they all uses this function, the Tail Recursion. At the end, we combine all the StyleCop Projects created from the source files, project files and solution files.

I could have created a counter function that has a Closure inside to count the ID’s though:

This would have reduced the arguments we must send with the function, and would remove the implicit reference to the project ids and the project length.

Feel free to contribute!

C# Assemblies

The assemblies used in this file, are also written in C#. So, this is an example of how C# assemblies can be used in F# files without much effort. The bad side is that a “list” in C# isn’t the same as a “list” in F#, so some conversions are needed.

Also, Currying or Partial Functioning in F# isn’t possible with C# objects. If this was possible, I think my implementation would look just that bit more Functional than now.

Conclusion

Personally, I was surprised that I could write a full implementation in just 2 - 3 hours. F# is a fun language to play with, but also to write solid declarative implementations quickly. I hope that I can use my functional skills more in actual production projects.

My interest in functional programming has increased by implementing an implementation for StyleCop and I expect that not only my functional programming skills will become better in the future but also my object-oriented programming skills.

Thanks for reading and check the implementation on my GitHub.

Remember that I only worked for 2 - 3 hours; so, contribute to the FAKE library if you have any suggestions because EVERYTHING can be written better.

Categories: Technology
Tags: Code Quality, F#
written by: Stijn Moreels

Posted on Tuesday, June 6, 2017 5:00 PM

Stijn Moreels by Stijn Moreels

Several things became clear to me when studying CI. One of things is that everything is based on the principle of automation. The moment when you start thinking about “I can’t automate this”: that’s the moment when you should ask yourself if that is really the case.

Introduction

Before I read the book about Continuous Integration by Paul Duvall, Stephen M. Matyas III, and Andrew Glover, I thought that CI meant that we just create a deployment pipeline in which we can easily/automate deployment of our software. That and the fact that developers integrate continuously with each other.

I’m not saying that it’s a wrong definition, I’m saying that it might be too narrow for what it really is.

Thank you, Paul, Stephen, and Andrew, for the inspiring book and the motivation to write this post.

Automation

Several things became clear to me when studying CI. One of these things is that everything is based on the principle of automation. The moment when you start thinking about “I can’t automate this” that’s the moment when you should ask yourself if that is really the case.

CI is all about automation. We automate the Compilation (different environments, configurations), Testing (unit, component, integration, acceptance…), Inspection (coverage, complexity, technical debt, maintainability index…), Documentation (code architecture, user manual, schemas…).

We automate the build that runs all these steps, we automate the feedback we get from it, …

You can almost automate everything. Things that you can’t automate are for example Manual Testing. The reason is that the definition of manual testing is that you let a human test your software. You let the human decide what to test. You can in fact automate the environment in which this human must test the software, but not the testing itself (otherwise it wouldn’t be called “manual” testing).

That’s what most intrigued me when studying CI - the automation. It makes you think of all those manual steps you must take to get your work done. All those tiny little steps that by itself aren’t meaning much but are a big waste if you see them all together.

If you always must build your software locally before committing, could we than just place the commit commands at the end of our build script?

Building

It’s kind of funny when people talk about “building” software. When some people say: “I can’t build the software anymore”; don’t always mean “build”; they mean “compile”. In the context of Continuous Integration, the “compile” step is only the first step of the pipeline but it’s sometimes the most important step to people. Many think of it as:

“If it compiles == it works”

When you check out some code and the Build fails (build, not compilation); that could mean several things: failed Unit Tests, missing Code Coverage, maximum Cyclometric Complexity, … but also a compilation failure.

In the next paragraphs, when I talk about a “build” I’m talking in the context of CI and don’t mean “compile”.

Continuous Building Software

Is your build automated?
Are your builds under 10 minutes?
Are you placing the tasks that will most likely to fail at the beginning of your build?
How often do you run your integration builds? Daily? Weekly? At every change (continuously)?

  • Every developer should have the ability to run (on demand) a Private Build on his or her machine.
  • Ever project should have the ability to run (on demand, polled, event-driven) an Integration Build that include slower tasks (integration/component tests, performance/load tests…),
  • Every project should have the ability to run (on demand, scheduled) a Release Build to create deployable software (typically at the end of the iteration), but must include the acceptance tests.

There are tremendous build script tools available to automate these kinds of things. NAnt, Psake, FAKE, Cake… are a few (I use FAKE).

Continuous Preventing Development/Testing

Are your tests automated?
Are you writing a test for every defect?
How many asserts per test? Limit to one?
Do you categorize your tests?

“Drive to fix the defect and prevent from reoccurring”

Many other posts discus the Test-First and Test-Driven mindset and the reason behind that; so, I will not discuss this here. What I will discuss is the reaction people have on a failing test from your build.

A failed build should trigger a “Stop the presses” event within the team. Everyone should be concerned about the failure and should help each other to make the build succeed again as quickly as possible. Fixing a failed build should be the responsible of the team and not (only) the person that broke the build.

But what do you do when the build failed? What reaction should you have?

First, write a test that exposes the defect by writing a test that passes. When that new test passes, you have proven the defect and can start fixing it. Note that we don’t write a failed test!

There are three reasons why you should write a test that passes for a defect (we’re using Test-Driven Development, right?):

  1. It’s difficult to write a failing test that uses the assertion correctly because the assertion may not be added when the test doesn’t fail anymore which means you don’t have a test that passes but a test that’s just not failing.
  2. You’re guessing what the fix should alter in behavior == assumption.
  3. If you have to fix the code being tested, you have a failing test that works but one that doesn’t verify the behavioral change.

To end the part of testing, let me be clear on some points that many developers fail to grasp: the different software tests. I have encountered several definitions of the tests so I merge them here for you. I think the most important part is that you test all these kind of aspects and not the part if you choose to call your Acceptance Tests, or Functional Tests:

  • Unit Tests: testing the smallest possible “units” of code with no external dependencies (including file system, database…), written by programmers - for programmers, specify the software at the lowest level…
    Michael Feathers has some Unit Test Rulz that specify whether a test can be seen as a Unit Test.
  • Component Tests encapsulate business rules (could include external dependencies), …
  • Integration Tests don’t encapsulate business rules (could include external dependencies), tests how components work together, Plumbing Tests, testing architectural structure, …
  • Acceptance Tests (or Functional Tests) written by business people, define the definition of “done”, purpose to give clarity, communication, and precision, test the software as the client expects it, (Given > When > Then structure), …
  • System Tests test the entire system, could sometimes overlap with the acceptance tests, test the system in a developer perspective…

Continuous Inspection

Can you show the current amount of code complexity?
Performing automated design reviews?
Monitoring code duplication?
Current code coverage?
Produce inspection reports?

It wouldn’t surprise you that Code Inspection is maybe not the most “sexy” part of software development (is Code Testing sexy?). But nonetheless it’s a very important part of the build.

Try to ask some projects what their current Code Coverage is, Maintainability Index? Technical Debt? Duplication? Complexity?...

Al those elements are so easily automated but so little teams adopt this mindset of Continuous Inspection. These elements are a certain starting point:

Continuous Deployment

Can you rollback a release?
Are you labelling your builds?
Deploy software with a single command?
Deploy with different environments (configuration)?
How do you handle fixes after deployment?

At the end of the pipeline (in a Release Build), you could trigger the deployment of the project. Yes, you should include the Acceptance Tests in here because this is the last step before the actual deployment.

The deployment itself should be done with one “Push on the Button”; as simple as that. In Agile projects, the deployment of the software is already done at the very beginning of the project. This means that the software is placed at the known deployment target as quickly as possible.

That way the team get as quickly as possible feedback of how the software act in “the real world”.

Continuous Feedback

When you deploy, build, test, … something, wouldn’t you want to know as quickly as possible what happened? I certainly do.

One of the first things I always do when starting a project is checking if I (and the team) gets the right notifications. As a developer, I want to know as quickly as possible when a build succeeds/failed. As an architect, you want to know what the current documentation of the code base is and what the code looks like in schemas, as project manager you may want to know if the acceptance tests where succeeded so the clients get what he/she wants…

Each function has its own responsibilities and its own reason to want feedback on things. You should be able to give them this feedback!

I use Catlight for my build feedback, work item tracking, release status... This tool will maybe in the future support pull request notifications too.

Some development teams have an actual big colorful lamp that indicate the current build status. Red = Failed, Green = Successful and Yellow = Investigating. Some lamps go more lighter/darker red if the build states in a "failed" state for too long.

Conclusion

Don’t call this a full-CI summary because it is certainly not. See this as a quick introduction of how CI can be implemented in a software project with the high-level actions in place and what you can improve in your project automation process. My motto is that anything can be improved and so, be more automated.

I would also suggest you read the book I talked about and/or check the site of Thought Works for more information on the recent developments in the CI community.

Start integrating your software to develop software with lesser risk and higher quality. Make it as automated that you just must “Push the Button”The Integrate Button.

Categories: Technology
written by: Stijn Moreels