wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Tuesday, June 6, 2017 5:00 PM

Stijn Moreels by Stijn Moreels

Several things became clear to me when studying CI. One of things is that everything is based on the principle of automation. The moment when you start thinking about “I can’t automate this”: that’s the moment when you should ask yourself if that is really the case.

Introduction

Before I read the book about Continuous Integration by Paul Duvall, Stephen M. Matyas III, and Andrew Glover, I thought that CI meant that we just create a deployment pipeline in which we can easily/automate deployment of our software. That and the fact that developers integrate continuously with each other.

I’m not saying that it’s a wrong definition, I’m saying that it might be too narrow for what it really is.

Thank you, Paul, Stephen, and Andrew, for the inspiring book and the motivation to write this post.

Automation

Several things became clear to me when studying CI. One of these things is that everything is based on the principle of automation. The moment when you start thinking about “I can’t automate this” that’s the moment when you should ask yourself if that is really the case.

CI is all about automation. We automate the Compilation (different environments, configurations), Testing (unit, component, integration, acceptance…), Inspection (coverage, complexity, technical debt, maintainability index…), Documentation (code architecture, user manual, schemas…).

We automate the build that runs all these steps, we automate the feedback we get from it, …

You can almost automate everything. Things that you can’t automate are for example Manual Testing. The reason is that the definition of manual testing is that you let a human test your software. You let the human decide what to test. You can in fact automate the environment in which this human must test the software, but not the testing itself (otherwise it wouldn’t be called “manual” testing).

That’s what most intrigued me when studying CI - the automation. It makes you think of all those manual steps you must take to get your work done. All those tiny little steps that by itself aren’t meaning much but are a big waste if you see them all together.

If you always must build your software locally before committing, could we than just place the commit commands at the end of our build script?

Building

It’s kind of funny when people talk about “building” software. When some people say: “I can’t build the software anymore”; don’t always mean “build”; they mean “compile”. In the context of Continuous Integration, the “compile” step is only the first step of the pipeline but it’s sometimes the most important step to people. Many think of it as:

“If it compiles == it works”

When you check out some code and the Build fails (build, not compilation); that could mean several things: failed Unit Tests, missing Code Coverage, maximum Cyclometric Complexity, … but also a compilation failure.

In the next paragraphs, when I talk about a “build” I’m talking in the context of CI and don’t mean “compile”.

Continuous Building Software

Is your build automated?
Are your builds under 10 minutes?
Are you placing the tasks that will most likely to fail at the beginning of your build?
How often do you run your integration builds? Daily? Weekly? At every change (continuously)?

  • Every developer should have the ability to run (on demand) a Private Build on his or her machine.
  • Ever project should have the ability to run (on demand, polled, event-driven) an Integration Build that include slower tasks (integration/component tests, performance/load tests…),
  • Every project should have the ability to run (on demand, scheduled) a Release Build to create deployable software (typically at the end of the iteration), but must include the acceptance tests.

There are tremendous build script tools available to automate these kinds of things. NAnt, Psake, FAKE, Cake… are a few (I use FAKE).

Continuous Preventing Development/Testing

Are your tests automated?
Are you writing a test for every defect?
How many asserts per test? Limit to one?
Do you categorize your tests?

“Drive to fix the defect and prevent from reoccurring”

Many other posts discus the Test-First and Test-Driven mindset and the reason behind that; so, I will not discuss this here. What I will discuss is the reaction people have on a failing test from your build.

A failed build should trigger a “Stop the presses” event within the team. Everyone should be concerned about the failure and should help each other to make the build succeed again as quickly as possible. Fixing a failed build should be the responsible of the team and not (only) the person that broke the build.

But what do you do when the build failed? What reaction should you have?

First, write a test that exposes the defect by writing a test that passes. When that new test passes, you have proven the defect and can start fixing it. Note that we don’t write a failed test!

There are three reasons why you should write a test that passes for a defect (we’re using Test-Driven Development, right?):

  1. It’s difficult to write a failing test that uses the assertion correctly because the assertion may not be added when the test doesn’t fail anymore which means you don’t have a test that passes but a test that’s just not failing.
  2. You’re guessing what the fix should alter in behavior == assumption.
  3. If you have to fix the code being tested, you have a failing test that works but one that doesn’t verify the behavioral change.

To end the part of testing, let me be clear on some points that many developers fail to grasp: the different software tests. I have encountered several definitions of the tests so I merge them here for you. I think the most important part is that you test all these kind of aspects and not the part if you choose to call your Acceptance Tests, or Functional Tests:

  • Unit Tests: testing the smallest possible “units” of code with no external dependencies (including file system, database…), written by programmers - for programmers, specify the software at the lowest level…
    Michael Feathers has some Unit Test Rulz that specify whether a test can be seen as a Unit Test.
  • Component Tests encapsulate business rules (could include external dependencies), …
  • Integration Tests don’t encapsulate business rules (could include external dependencies), tests how components work together, Plumbing Tests, testing architectural structure, …
  • Acceptance Tests (or Functional Tests) written by business people, define the definition of “done”, purpose to give clarity, communication, and precision, test the software as the client expects it, (Given > When > Then structure), …
  • System Tests test the entire system, could sometimes overlap with the acceptance tests, test the system in a developer perspective…

Continuous Inspection

Can you show the current amount of code complexity?
Performing automated design reviews?
Monitoring code duplication?
Current code coverage?
Produce inspection reports?

It wouldn’t surprise you that Code Inspection is maybe not the most “sexy” part of software development (is Code Testing sexy?). But nonetheless it’s a very important part of the build.

Try to ask some projects what their current Code Coverage is, Maintainability Index? Technical Debt? Duplication? Complexity?...

Al those elements are so easily automated but so little teams adopt this mindset of Continuous Inspection. These elements are a certain starting point:

Continuous Deployment

Can you rollback a release?
Are you labelling your builds?
Deploy software with a single command?
Deploy with different environments (configuration)?
How do you handle fixes after deployment?

At the end of the pipeline (in a Release Build), you could trigger the deployment of the project. Yes, you should include the Acceptance Tests in here because this is the last step before the actual deployment.

The deployment itself should be done with one “Push on the Button”; as simple as that. In Agile projects, the deployment of the software is already done at the very beginning of the project. This means that the software is placed at the known deployment target as quickly as possible.

That way the team get as quickly as possible feedback of how the software act in “the real world”.

Continuous Feedback

When you deploy, build, test, … something, wouldn’t you want to know as quickly as possible what happened? I certainly do.

One of the first things I always do when starting a project is checking if I (and the team) gets the right notifications. As a developer, I want to know as quickly as possible when a build succeeds/failed. As an architect, you want to know what the current documentation of the code base is and what the code looks like in schemas, as project manager you may want to know if the acceptance tests where succeeded so the clients get what he/she wants…

Each function has its own responsibilities and its own reason to want feedback on things. You should be able to give them this feedback!

I use Catlight for my build feedback, work item tracking, release status... This tool will maybe in the future support pull request notifications too.

Some development teams have an actual big colorful lamp that indicate the current build status. Red = Failed, Green = Successful and Yellow = Investigating. Some lamps go more lighter/darker red if the build states in a "failed" state for too long.

Conclusion

Don’t call this a full-CI summary because it is certainly not. See this as a quick introduction of how CI can be implemented in a software project with the high-level actions in place and what you can improve in your project automation process. My motto is that anything can be improved and so, be more automated.

I would also suggest you read the book I talked about and/or check the site of Thought Works for more information on the recent developments in the CI community.

Start integrating your software to develop software with lesser risk and higher quality. Make it as automated that you just must “Push the Button”The Integrate Button.

Categories: Technology
written by: Stijn Moreels

Posted on Thursday, June 1, 2017 3:30 PM

Stijn Moreels by Stijn Moreels

One of the mind-blowing development techniques that radically changed the programming world is the Test-Driven Development approach introduced by Kent Beck.

But... writing tests before we start coding? Who will do that?

Introduction

One of the mind-blowing development techniques that radically changed the programming world; is the Test-Driven Development.

Writing tests before we start coding? Who will do that?

I must admit that I personally wasn’t really convinced by the idea; maybe because I didn’t quite understand the reason we should write our tests first and the way we should do it. Can you have a bad Software Design with TDD? Can you break your Architecture with TDD? Yes! TDD is a discipline that you should be following. And like any discipline you must hold to a certain number of requirements. By the end of the day, it’s YOUR task to follow this simple mindset.

In this article, I will talk about the Mindset introduced by Kent Beck when writing in a Test-Driven Development environment.

Too many developers don’t see the added-value about this technique and/or don’t believe it works.

TDD works!

"Testing is not the point; the point is about Responsibility"
-Kent Beck

Benefits

Because so many of us don’t see the benefits of TDD, I thought it would make sense to specify them for you. Robert C. Martin has inspired me with this list of benefits.

Increased Certainty

One of the benefits is that you’re certain that it works. Users have more responsibility in a Test-Driven team; because they will write the Acceptance Tests (with help of course), and they will define what the system must do. By doing so, you’re certain that what you write is what the customer wants.

The amount of uncertainty that builds up by writing code that isn’t exactly what the customer wants is called: The Uncertainty Principle. We must always eliminate this uncertainty.

By writing tests first; you can tell your manager and customer: “Yes, it will work; yes, it’s what you want”.

Defect Reduction

Before I write in a Test-First mindset; I always thought that my code was full of bugs and doesn’t handle unexpected behavior.
Maybe it was because I’m very certain of myself; but also, because I wrote the tests after the code and was testing what a just wrote; not what I want to test.

This increases the Fake Coverage of your code.

Increased Courage

So many developers are “afraid” to change something in their code base. They are afraid to break something. Why are they afraid? Because they don’t have tests!

“When programmers lose the fear of cleaning; they clean”
- Robert C. Martin

A professional developer doesn’t allow that his/her code rots; so, you must refactor with courage.

In-Sync Documentation

Tests are the lowest form of documentation of your code base; always 100% in sync with the current implementation in the production code.

Simple Design

TDD is an analysis/design technique and not necessary a development technique. Tests force you to think about good design. Certainly, if you write them BEFORE you write the actual implementation. If you do so, you’re writing them in offence and not in defense (when you’re writing them afterwards).

Test-First also helps you think about the Simplest thing that could possibly work which automatically helps you to write simple structured designed code.

Test-First Mindset

When you’re introduced into the Test-First methodology, people often get Test Infected. The amount of stress that’s taking from you is remarkable. You refactor more aggressively your code without any fear that you might break something.

Test-Driven Development is based on this very simple idea to first write your test, and only then write your production code. People underestimate the part “first write your test”. When you writing your tests, you’re solving more problems than you think.

Where should I place this code? Who’s responsible for this logic? What names should I use for my methods, classes…? What result must a get from this? What isn’t valid data? How will my class interface look like? …

After trying to use TDD in my daily practice, if found myself always asking the same questio:

“I would like to have a … with … and …”

Such a simple idea changed my vision so radically about development and I’m convinced that by using this technique, you’re writing simpler code because you always think about:

“What’s the simplest thing that could make this test work”

If you find that you can implement something that isn’t the right implementation, write another test to expose this behavior you want to implement.

TDD is - in a way - a physiological methodology. What they say is true: you DO get addicted to that nice green bar that indicate that you’re tests all pass. You want that color as green as possible, you want it always green, you want it run as fast as possible so you can quickly see it’s green…

To be a Green-Bar-Addict is a nice thing.

Kent Beck Test-Patterns

It felt a little weird to just state all the patterns Kent Beck introduced. Maybe you should just read the book Test-Driven Development by Example; he’s a very nice writer and I learned a lot from the examples, patterns and ideas.

What I will do, is give you some basic patterns I will use later in the example and some patterns that we very eye-opening for me the first time.

Fake It

When Kent talked about “What’s the simplest thing that could work”, I was thinking about my implementation but what he meant was “What’s the simplest thing that could work for this test”.

If you’re testing that 2 x 3 is 6 than when you implement it, you should Fake It and just return the 6.

Very weird at first, especially because the whole Fake It approach is based on duplication; the root of all software evil. Maybe that’s the reason experienced software engineers are having problems with this approach.

But it’s a very powerful approach. Using this technique, you can quickly get the bar green (testing bar). And the quicker you get that bar green, the better. And if that means you must fake something; then you should do that.

Triangulation

This technique I found very interesting. This approach really drives the abstraction of your design. When you find yourself not knowing what to do next, or how you should go further with your refactoring; write another test to support new knowledge of the system and the start of new refactorings in your design.

Especially when you’re unsure what to do next. 

If you’re testing that 2 x 3 is 6 than in a Triangulation approach you will first return 6 and only change that if you’re testing again but then for 2 x 2 is 4.

Obvious Implementation

Of course: when the implementation is so simple, so obvious, … Than you could always implement it directly after your test. But remember that this approach is only the second option after Fake It and Triangulation.

When you find yourself taking steps that are too big, you can always take smaller steps.

If you’re testing that 2 x 3 is 6, in an Obvious Implementation approach you will just write 2 x 3 right away.

By Example

I thought it would be useful to show you some example of the TDD workflow. Since everyone is so stoked about test-driving Fibonacci I thought it would be fun to test-drive another integer sequence.

Let’s test-drive the Factorial Sequence!

What happens when we factorial 4 for example? 4! = 4 x 3 x 2 x 1 = 24

Test

But let’s start with something super simple:

Always start with the same sentence: “I would like to have a… “. I would like to have a method called Factorial which I could use to send an integer with that will calculate the factorial integer for me.

Now we have created a test before anything about factorial is implemented.

Compile

Now we have the test, let’s start now by making our code compile again.

Let’s test this:

Hooray! We have a failed test == progress!

Implement

First Steps

What’s the simplest thing that we could write in order that this test will run?

Hooray! Our test passed, we can go home, right?

A Bit Harder

What’s next? Let’s check. What happens if we would test for another value?

I know, I know. Duplication, duplication, duplication. But were testing now right, not yet in the last step of the TDD mantra.

What is the simplest we could change to make this test pass?

Yes, I’m feeling good right now. A nice green bar.

One Step Before Generalize

Let’s add just another test, a bit harder this time. But these duplication is starting to irritate me; you know the mantra: One-Two-Three-Refactor? This is the third time, so let’s start refactoring!

Ok, what’s the simplest thing?

Generalize

Ok, we could add if/else-statements all day long, but I think it’s time to some generalization. Look at what we’ve now been implementing. We write 24 but do we mean 24?

Remembering Factorial, we mean something else:

All still works, yeah. Now, we don’t actually mean 4 by 4 do we. We actually mean the original number:

And we don’t actually mean 3, 2, and 1 by 3, 2 and 1, we actually mean the original number each time mins one. So, actually the Factorial of the 3! could you say, no?

Let’s try:

Wow, still works.Wait, isn’t that if-statement redundant? 2 x 2! == 2 right?

Exploration

Now, the factorial of 0, is also 1. We haven’t tested that haven’t we? We have found a boundary condition!

This will result in a endless loop because we will try to factorial an negative number; and since factorial only happens with positieve numbers (because the formula with negative integers will result in a division by zero and so, blocking us for calculating a factorial value for these negative integers).

Again, simplest thing that could work?

Now, the last step of TDD is always Remove Duplication which in this case is the 1 that’s used two times. Let’s take care of that:

Hmm, someone may have noticed something. We could actually remove the other if-statement with checking for 1 if we adapt the check of 0. This will return 1 for us in the recursive call:

By doing this, we also have ruled out all the other negative numbers passed inside the method.

Conclusion

Why oh why are people so skeptic about Test-Driven Development. If you look at the way you use it in your daily practice, you find yourself writing simpler and more robust code.

TDD is actually a Design Methodology and not a Development Methodology. The way you think about the design, the names, the structure… all that is part of the design process of your project. The tests that you have is the added value of this approach and makes sure that you can refactor safely and are always certain of your software.

Start trying today in your daily practice so you stop thinking about: How will you implement it? but rather:

How will you test it?

 

Categories: Technology
written by: Stijn Moreels

Posted on Thursday, March 30, 2017 1:56 PM

Stijn Moreels by Stijn Moreels

When people talk about Code Style they refer to their Naming Conventions, Code Structure…
but what they sometimes forget are the Formatting Guidelines. Something that’s not been taken serious enough in my opinion; but it’s one of the guidelines that I like very much to talk about.

Introduction

When people talk about Code Style they refer to their Naming Conventions, Code Structure…
What they tend to forget are the Formatting Guidelines. Something that’s not been taken serious enough in my opinion; but it’s one of the guidelines that I like to talk about.

With Formatting Guidelines, I mean the rules and guidelines that have been adopted in your team to format your code base files. When I hear people question my vision about these guidelines; that’s the moment when I’m certain that it’s INDEED important to adopt Formatting Guidelines for your team.

The reason people tend to question this, is because they don’t see the purpose and reason why these guidelines are important. I am convinced that Formatting in the topic of Code Style is important.

Formatting Basics

Formatting?

What do I mean with Formatting? Isn’t that just wasteful work? It doesn’t change my code behavior?
The first question I will discuss in this paragraph; and NO, it isn’t wasteful work; and NO, it doesn’t change your codes behavior.

What formatting means is the structure of your code. Just like adding an ENTER statement to start a new paragraph, or adding a trailing space after the point of your sentence.

Formatting is all about these little modifications in your code base. We developed a Formatting Guidelines document which captures our teams’ guidelines involving all the different ways we format our code. This document isn’t fixed but is dynamically formed and changes weekly by adding more and more verified ideas to format our code.

Formatting is about changing your code in a uniform way to make your intensions clear. Only a non-professional would ignore formatting and not use it in his/her daily practice.

Formatting Principles

One of the most important principles in Extreme Programming (XP) is Humanity. Maybe a simple principle at first but a very large portion of today’s literature doesn’t validate this simple fact:

“People develop software”
-Kent Beck

The Humanity together with Quality is for me the MOST important principles if you think about formatting in code style conventions. We as people develop software; we as people are reading this software daily; we as people are changing this software daily…

Wouldn’t it be more than normal that we create an environment where people feel safe to contribute, feel like they belong to the teams’ efforts and growth so they fully are being understood by everyone?

THAT’S what Humanity is all about. Formatting is about people. Formatting allows you to structure your code in a uniform way so that everyone can quickly spot what they are searching for. We, as developers, are reading more code than we write; so, with that in mind we should have our focus on the way people read code instead of just writing the code.

When I first heard about this statement, I was even more convinced that code should be: Simple, Clean, and Structured.

Formatting Patterns

The following patterns are just some basic ideas that I’d like to share with you. These aren’t all the patterns but I list them here to give you an idea to define your own patterns and come up with new ideas for your own.

Separate Concepts

It’s magical what just a single ENTER can do in code, but be careful where you use it. If you have a 10-line method and you place some ENTER here and there to increase Readability, that is a good thing, but maybe not the best solution.
Maybe you have the beginning of the Long Method Code Smell and have to refactor some with Extract Method Refactoring.

What I like to discuss are the other cases: the cases with where you have a method with a return value for example. Look at the following code snippet:

I could just have placed the “return new Attachment” statement with the two others, but instead I placed it on a separate line. Another example:

Here I have an ENTER after the Guard Clause; while I’ll just could have place the “UpdatePersonFromStore()” method a line up.

Both examples indicate the pattern of Separate Concepts. This patterns is about the separation of different concepts. The ENTER is the Separation in this context and the return/guard-clause are the Concepts.

The “return” statement in the first example is the actual Plot of the method. This plot is more important than the retrieval of the content type or location. It’s also a total different Concept, so we place it on a different line.

The second example with the if-statement is about time you spend reading. When there’s an exception being thrown, we won’t have to look at any other statements. So, we placed an ENTER after the exception statement.

These two examples are just a start for the practical implementation of the Separate Concepts pattern; but now you can find your own places to separate the concepts.

Conceptual Affinity

The second pattern I want to discuss is Conceptual Affinity. The relationships of elements of the same concept; that’s what this pattern means.

The practice is finding ways to place elements of the same concept as close as possible with each other.

We could just place the “SetLow(int)” method below the “Calculate()” method; but we didn’t do that because:

  1. SetHigh and SetLow are method with the same Signature
  2. SetHigh and SetLow are doing structurally the same action

With these two reasons, we can conclude that these two methods are more likely in the same Concept than the relationship with the "Calculate()" method.

A bit subtler is the following example. We could also have retrieved the one hour before we retrieved the current time; but to support the Concept of calculating the time from now with one hour, we have the retrieval logic in the same order as the formula.

This way the concept of calculating is related to the concept of retrieval and increases the Cleaniness and Simplicity of the code.

Try to switch the values and see for yourself that this way of writing is easier to follow.

Now it’s time to spot your concepts and the best solution. The last example of is on the edge of the refactoring Extract Method I think; if the calculation of the two values would become more complex, than that would be a good time to extract the two temporary variables to their own methods.

Newspaper Metaphor

The last pattern I will discuss with you is the Newspaper Metaphor (from the book Clean Code). This pattern talks about the global structure of your code files. The location of each element and the proper position.

When you write a class, try to hold this pattern in mind. Just like a newspaper, you place the Header and important items on the top, and place the Details and more concrete information at the bottom.

This will increase the readability of your code file; and you will spend less time searching the file.

Following structure could you use to define the order of the global elements:

  1. Fields
  2. Constructors
  3. Properties
  4. Public Methods

In each element, you could also define a structure for the visibility and place for example the Public Methods before the Private Methods.

Besides the global structure of elements is the Newspaper Metaphor useful for the flow of the code file. Just like a newspaper you must make sure that you can read your file from top to bottom without jumping (too much) back up. This is also called the Step-Down Rule.

Notice that this sequence is a standard convention of the Microsoft Guidelines and not an invented idea. Also, when writing in this structure always think of the relationship between components:
If a public method has several private methods which it delegates to, place those methods as close to the public method as possible. If you’re reading a headline in a newspaper and find it interesting, you want to read more of the details directly under the headline and not on a different page.

With this last pattern, you can start with searching for the perfect article in your code and make it worth the read.

Conclusion

When people don't find it very interesting or don’t see the added-value of Formatting; they don’t think in people-terms but in terms of tasks. They’re Task-Minded and not People-Minded.

I hope with this short article about Formatting, you will also be/become People-Minded and use these peoples as your teams’ strength.

There’s a lot more patterns about Formatting which increases readability and supports Humanity; but I’m sure that, with this as starting point, you will find your own patterns that make YOU more comfortable. Try to make a Formatting Guidelines document for your team so everyone feels comfortable about the whole Formatting practice and everyone can make sure that everyone’s daily development work is just that little bit more restful and less annoying.

My mistake was to write a long document with all the possible approaches I could think of; but not everyone is so disciplined in adopting these patterns so try to limit yourself in a One-Page-Document (Example in Java) with maybe a class as example to show:

  • How do you sort fields, members?
  • How to write flow statements (return, break, continue, …)?
  • Where do you place enters to increase readability?

Codit is a company which primary focus is People so it’s only logical that we are paying attention to this practice of Formatting and making sure that everyone feels comfortable about reading code.

 

“People develop software”

Categories: Architecture
written by: Stijn Moreels

Posted on Tuesday, February 28, 2017 4:28 PM

Stijn Moreels by Stijn Moreels

What do I mean with “Continuous Refactoring”? related terms are: Merciless Refactoring, The Boy Scout Rule… The main principle is that you refactor constantly without mercy to improve the design of your code. The boy scout rule state that “you must leave the playground cleaner than you found it”.

Some teams are still cautious about this practice because they see the risk of change; what they don’t see is the risk of keeping it and the cost of working around it.

Introduction

My favorite topic in software development is code quality; and to be specific: Refactoring.

Refactoring you ask? One of the very building blocks in software quality is refactoring. You can’t simply deliver a high-quality software product without refactoring; period. Since everyone is responsible of quality (everyone!); therefore, everyone is responsible to refactor their software.

What do I mean with “Continuous Refactoring”? related terms are: Merciless Refactoring, The Boy Scout Rule… The main principle is that you refactor constantly without mercy to improve the design of your code. The boy scout rule state that “you must leave the playground cleaner than you found it”.

Some teams are still cautious about this practice because they see the risk of change; what they don’t see is the risk of keeping it and the cost of working around it.

Always search for places that Smell; be critical about your work and call everything into question.

What the current culture of software design school is taught are the different Design Patterns, Principles, Practices, and other different solutions in software design to define solutions. But what sometimes is forgotten is that’s a lot more interesting and valuable to learn the problems instead of the solutions.

"See the Design Patterns in the context of Refactoring"
-Joshua Kerievsky, Refactoring to Patterns

“Refactoring allows us to change our mind”
-Martin Fowler, from Refactoring

“For each desired change, make the change easy, then make the easy change”
-Kent Beck

 Values, Principles, and Practices

Before I start explaining any further about Continuous Refactoring, I will explain you some required values, principles, and practices to perform refactoring.

Refactoring is not just “rewriting code” or anything like that. Some people think they refactor by changing some code, go through every error until it compiles and then think because it compiles they have done "refactoring”.

Refactoring is a process of well-structured steps, each by which tests play a fundamental role.

“Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which "too small to be worth doing"
-Marin Fowler, Refactoring

In the line of Extreme Programming (XP) I will go through some values, principles, and practices that I hold high.

"Driving is not about getting the car going in the right direction.
Driving is about constantly paying attention, making a little correction this way, a little correction that way"
-Kent Beck (Extreme Programming, Embrace Change)

Values

Communication is one of the values (maybe THE value) in Extreme Programming that’s important for the effective cooperation in a team environment. Here as well, refactoring plays a huge part. It’s by performing refactoring that you create more understanding of your code base.

This is called Comprehension Refactoring as one of the Workflows of Refactoring. By reading the code base; you change a variable name, you move function around, you inline a parameter…

By doing so, you’re starting to increasingly learn your code base and become familiar with it.

Another value I would like to discuss is Simplicity. “What is the simplest thing that could possibly work?”. At first this looks maybe a “simple” value; but I think it’s one of the hardest ones. When is something simple? When you could understand something without too much thinking? When it’s obvious?

A lot of refactoring patterns (low-level and high-level) have Simplicity in mind. Always find a way to make your code simpler, more obvious, more explicit... so anyone could immediately understand what your intensions are.

Simplicity only works in defined context: when something is simple for you, it may not be simple for your other team members, and therefore not Simple Enough. That’s also one of the excuses that people use to explain their bad code by making No Design as alternative of Big-Design-Up-Front; but you must use Little-Design- or Enough-Design-Up-Front.

They think Quick-And-Dirty is the same as Simple Design; but Quick-And-Dirty is an oxymoron, it means slow, not quick in the end. It’s by designing just enough and refactor every day that you increase the Simplicity of your software.

“The problem with quick and dirty is that dirty remains long after quick has been forgotten”
-Steve C McConnell, Software Project Survival Guide

Principles

One of the principles of XP I would discuss here is Collective Ownership. The code base is owned by the team. It’s a very bad practice to have an environment where everyone is “responsible” for their own piece of code. This would create barriers for each developer and a tunnel-vision in writing your software.

It’s by reviewing others that you will improve your own code. The purpose is that you create the simplest and most obvious software possible, and not just simple and obvious for yourself. The ONLY way this could work is that everyone (everyone!) write and run tests for their code.

Every one of the team can improve any part of the system at any time.

This may first feel uncomfortable when you change “someone else his/her code”; but that’s the problem. There is no his/her code, only our code. That’s the solution.

Other principles of XP are Quality, Baby Steps and Improvement which exactly making my point when I was talking about refactoring. People sometimes cut in quality to increase their project speed. (you know, the Quick-And-Dirty approach). Decreased quality will not increase the speed of your project; it’s just by increasing the quality that your project will have faster delivery. It’s with lower quality that you will have a less predictable and later delivery.

Refactoring talks about making “small behavior-preserving transformations” or Baby Steps that must undertake in your refactoring process. It may be very tempting to skip some steps, don’t run your tests after a baby step or do some steps together to have the illusion that you will be done in a lesser time. The opposite is true. Always think in the least step that you can take in the right direction; what is the smallest step you can take?

Renaming a variable? Moving a method? Extracting an interface? Pulling a field?...

Practices

What do you need to do to perform a valid refactoring? I’ll show you some practices.

The one I will always place forward first, is Automated Tests. Unit Tests should have a virtual code coverage percentage of 100% at all time of your code. In XP and other methodologies, you also need to define Acceptance Tests to perform story building tests at system-level.

Both type of tests is written first in a Test-First scenario. Refactoring is the third other part that define Test-Driven-Development (TDD).

Test-First + Refactoring = Test-Driven-Development

You first write a failing unit test at the beginning of your implementation, and you first write a failing acceptance test in the beginning of your agile iteration. That’s ideally. Some write their tests after implementation in a Test-Least-Development approach; but I think that every respectable developer thinks about the tests while developing and whether you write first your tests and then the implementation or write directly after your implementation your tests: in both cases, you think of how your production code will be tested and that’s the purpose of a Test-First approach, you’re only doing it backwards in a Test-Least Development scenario.

So, I leave it up to you what approach you will use, but I feel more comfortable when I writing my tests first.

Test – Code – Refactor

Another practice that you will absolutely need is Continuous Integration. By continuously integrating with your team members, will decrease unpredictability and the cost of integration. Develop your features in a different branch, but always integrate in a couple of hours. It’s a bad practice if you can’t point out a valuable reason to have multiple code streams, you’ll have an enormous waste in your software.

Instead of having continuously multiple streams, fix your underlying design problem by refactoring.

How, Why, and When?

Now that I give you a quick introduction, I’ll show you some basic concepts around Continuous Refactoring.

How Should You Refactor?

The easiest one is the How. How should you refactor? That’s rather simple, Martin Fowler and Joshua Kerievsky has started the refactor pattern story by creating a starting point of what the mechanics are in a refactoring environment.

Martin talks about low-level refactorings like Move Method, Encapsulate Field, Replace Inheritance with Delegation, Decompose Conditional, Encapsulate Collection… while Joshua talks about composed and test-driven refactorings like Replace Implicit Tree with Composite, Unify Interfaces with Adapter, Encapsulate Composite with Builder

A lot of these refactoring patterns are based on a pattern Expand – Migrate – Contract which allows Parallel Change and backwards compatibility. You should always hold in mind that during every step, EVERYTHING MUST STILL WORK. The pattern of Expand – Migrate – Contract tells you to expand your code base always first instead of directly change your code. Because you first add new code instead of adapting existing; you create an environment of backwards-compatible components. Just like unit tests are given you confidence, so does this pattern.

As example I can talk about a “simple” refactoring pattern Add Parameter from Martin. This pattern describes how to add a parameter to a method. Instead of just adding the parameter to the existing method, you first create a new method with this added parameter (Expand phase). After that, you change every call to the old method to the new one (Migrate phase). And as last, you remove the old method so the new method is now fully integrated (Contract phase).

After each phase, you compile/test, but also after each change from method calls. The process described in here for Add Parameter is how you refactor. (NOTE: C#, Java, Python… has the possibility to add default-value parameters to methods, which would help you with this refactoring. But I find it always useful to know more than a single solution for a problem).

What’s important is that you take Baby Steps in your refactoring process and slowly go to, towards or away from a current implementation choice by testing after every transformation step.

Why Should You Refactor?

It bit harder one to explain (especially to non-technical people) is the Why. Why should you refactor? Try to explain to a customer or non-technical manager that you will change your software so it will have more quality, and it would be easier to maintain and expand without there’s anything that will be changed or added of the current implemented functionality.

“If you aren’t going to change or add anything, could we just add the next feature?”

Business people think in features. Features added, removed, changed… but never refactored. The very definition of refactoring is that you are not going to change any outside observable behavior.

There is a way, though, that you can use to explain what “refactoring” really means to those people. It calls Technical Debt (synonym for Code Debt or Design Debt). Technical (or code or design) debt describes the virtual debt you must repay if you don’t decrease your debt. This is often called the Financial Metaphor and is a valuable solution to explain to non-technical people what the cost of not-refactoring means.

The next time you must defend yourself against someone who doesn’t see the risk your taking by NOT refactoring, pull out your credit card and tell them: “Would you skip the payment of your bills every month?”. Just like your own financial bills that you must pay to get back on track, so must you spent time refactoring so you don’t get in a situation that you can’t pay back your Technical (Code/Design) Debt.

Why should we refactor? Because we just can’t have THE perfect solution in the begin of the project with Up-Front Design so we must refactor daily to get each time closer to THE solution.

There are many reasons why you should refactor: change/adding/removing a feature will become easier, people will understand more quickly your design, bugs will experience more difficulty to hide, your code will remain fresh (without smells), …

When Should You Refactor?

"Seeing the trouble spot is often the hardest and most uncertain part"
- Eric Evans, Domain-Driven Design

The When is probably the hardest one. It’s a very lame answer but experience is the long-way to gain knowledge of the when; and you only gain experience by learning from making right and wrong decisions.

Reading books about refactoring (I’ve learned a lot from Martin Fowler, Joshua Kerievsky, Robert C. Martin; a.k.a. Uncle Bob…), Continuously Refactor and; fail. That’s the way you will start to learn when to refactor. Most design problems come when your design is:

  • Duplicated, implicit or explicit
  • Unclear, don’t show codes intent
  • Too complicated, over-engineered, not the simplest thing that could work

To spot places that need refactoring, you can start with learning different types of Code Smells. These smells will indicate that something is strange about the design, but hold on. The smells are only indicators and are not always the problem itself but only signals that there may be a problem. 

Some smells from Martin that are introduced in his Refactoring book: Feature Envy, Shotgun Surgery, Primitive Obsession, Refused Bequest… By building a vocabulary of refactorings patterns and code smells, you can quicker communicate with your team members about problem indicators you’ve found and solutions you propose.

You can also start introduce a “Code Smell Day” (typically at the end of each iteration) by which each team members starts looking in the code base at places that Smell. Or you can introduce a Refactoring Day in which you only refactor all the left over TODO items that needs to be refactored so your debt is repaid.

That’s for the technical part; the other part is about planning and time management. If you spot a candidate for refactoring; I find it always hard to place it in the scale of Reporting/Not-Reporting.

When the refactoring is about the feature you’re implementing and it will benefit the further process; then there’s no doubt, than you MUST refactor. If on the other hand, you find some smells in other parts of the system and it has nothing to do with what you’re doing. It could be useful to have a Public Refactor List by hand to describe what you’ve found and discuss it at the Weekly Meeting of the iteration so it can be a part of the next iteration.

OR you can refactor it right away and tell everyone of the team what you’re doing. Continuous Integration plays a big role in this one.

What I was saying about the Reporting/Not-Reporting part, is about the scale of your refactoring. If you refactor a method by performing Rename Method on it, or removing a temporary variable by executing Inline Temp... it would be an overload to define a work item for each refactoring; it should be a part of your development work. Just like you don’t specify for each feature a task “Writing Unit Tests”, you should not specify items with this low-level refactorings (even composite or test-driven refactorings). It should be a part of your work. Period.

But if the refactoring is a chain of multiple low-level, composite, or test-driven refactorings; than it smells more like a Design Refactoring and could it be of use to add this to the global planning as a story.

I find it sometimes difficult to know when you should Report the refactoring or Not. Should it be a Planned Refactoring? Should it be a Long-Term Refactoring?

The more technical Refactoring Workflows are already part of the Development Workflow; so, it would be redundant to specify these in the planning.

Conclusion

Refactoring is nowadays widely adopted in teams as a natural thing; although there’s still so many misunderstanding about the topic and what risks you and your team takes by NOT Continuously Refactoring. It should be a part of your development process; just like writing Unit- and Acceptance- Tests before implementing.

Practice your Continuous Refactoring skills daily and don’t be afraid to change your code daily. It’s a lot riskier to leave a dirty piece of code and to make workarounds every day than to change it so everyone can feel more comfortable.

Always think you’re programming in clay and not in stone: with some water, you can always continuous change the structure of your clay; while water only destroys stone.

Refactoring removes Duplication, increases Readability, Simplicity, and the Code’s Intent

So, start refactoring your project continuously and without any mercy for any written piece of code. When this is your daily practice; you may call yourself a Merciless Refactorer.

Categories: Architecture
written by: Stijn Moreels