wiki

Codit Wiki

Loading information... Please wait.

Codit Blog

Posted on Tuesday, February 28, 2017 4:28 PM

Stijn Moreels by Stijn Moreels

What do I mean with “Continuous Refactoring”? related terms are: Merciless Refactoring, The Boy Scout Rule… The main principle is that you refactor constantly without mercy to improve the design of your code. The boy scout rule state that “you must leave the playground cleaner than you found it”.

Some teams are still cautious about this practice because they see the risk of change; what they don’t see is the risk of keeping it and the cost of working around it.

Introduction

My favorite topic in software development is code quality; and to be specific: Refactoring.

Refactoring you ask? One of the very building blocks in software quality is refactoring. You can’t simply deliver a high-quality software product without refactoring; period. Since everyone is responsible of quality (everyone!); therefore, everyone is responsible to refactor their software.

What do I mean with “Continuous Refactoring”? related terms are: Merciless Refactoring, The Boy Scout Rule… The main principle is that you refactor constantly without mercy to improve the design of your code. The boy scout rule state that “you must leave the playground cleaner than you found it”.

Some teams are still cautious about this practice because they see the risk of change; what they don’t see is the risk of keeping it and the cost of working around it.

Always search for places that Smell; be critical about your work and call everything into question.

What the current culture of software design school is taught are the different Design Patterns, Principles, Practices, and other different solutions in software design to define solutions. But what sometimes is forgotten is that’s a lot more interesting and valuable to learn the problems instead of the solutions.

"See the Design Patterns in the context of Refactoring"
-Joshua Kerievsky, Refactoring to Patterns

“Refactoring allows us to change our mind”
-Martin Fowler, from Refactoring

“For each desired change, make the change easy, then make the easy change”
-Kent Beck

 Values, Principles, and Practices

Before I start explaining any further about Continuous Refactoring, I will explain you some required values, principles, and practices to perform refactoring.

Refactoring is not just “rewriting code” or anything like that. Some people think they refactor by changing some code, go through every error until it compiles and then think because it compiles they have done "refactoring”.

Refactoring is a process of well-structured steps, each by which tests play a fundamental role.

“Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which "too small to be worth doing"
-Marin Fowler, Refactoring

In the line of Extreme Programming (XP) I will go through some values, principles, and practices that I hold high.

"Driving is not about getting the car going in the right direction.
Driving is about constantly paying attention, making a little correction this way, a little correction that way"
-Kent Beck (Extreme Programming, Embrace Change)

Values

Communication is one of the values (maybe THE value) in Extreme Programming that’s important for the effective cooperation in a team environment. Here as well, refactoring plays a huge part. It’s by performing refactoring that you create more understanding of your code base.

This is called Comprehension Refactoring as one of the Workflows of Refactoring. By reading the code base; you change a variable name, you move function around, you inline a parameter…

By doing so, you’re starting to increasingly learn your code base and become familiar with it.

Another value I would like to discuss is Simplicity. “What is the simplest thing that could possibly work?”. At first this looks maybe a “simple” value; but I think it’s one of the hardest ones. When is something simple? When you could understand something without too much thinking? When it’s obvious?

A lot of refactoring patterns (low-level and high-level) have Simplicity in mind. Always find a way to make your code simpler, more obvious, more explicit... so anyone could immediately understand what your intensions are.

Simplicity only works in defined context: when something is simple for you, it may not be simple for your other team members, and therefore not Simple Enough. That’s also one of the excuses that people use to explain their bad code by making No Design as alternative of Big-Design-Up-Front; but you must use Little-Design- or Enough-Design-Up-Front.

They think Quick-And-Dirty is the same as Simple Design; but Quick-And-Dirty is an oxymoron, it means slow, not quick in the end. It’s by designing just enough and refactor every day that you increase the Simplicity of your software.

“The problem with quick and dirty is that dirty remains long after quick has been forgotten”
-Steve C McConnell, Software Project Survival Guide

Principles

One of the principles of XP I would discuss here is Collective Ownership. The code base is owned by the team. It’s a very bad practice to have an environment where everyone is “responsible” for their own piece of code. This would create barriers for each developer and a tunnel-vision in writing your software.

It’s by reviewing others that you will improve your own code. The purpose is that you create the simplest and most obvious software possible, and not just simple and obvious for yourself. The ONLY way this could work is that everyone (everyone!) write and run tests for their code.

Every one of the team can improve any part of the system at any time.

This may first feel uncomfortable when you change “someone else his/her code”; but that’s the problem. There is no his/her code, only our code. That’s the solution.

Other principles of XP are Quality, Baby Steps and Improvement which exactly making my point when I was talking about refactoring. People sometimes cut in quality to increase their project speed. (you know, the Quick-And-Dirty approach). Decreased quality will not increase the speed of your project; it’s just by increasing the quality that your project will have faster delivery. It’s with lower quality that you will have a less predictable and later delivery.

Refactoring talks about making “small behavior-preserving transformations” or Baby Steps that must undertake in your refactoring process. It may be very tempting to skip some steps, don’t run your tests after a baby step or do some steps together to have the illusion that you will be done in a lesser time. The opposite is true. Always think in the least step that you can take in the right direction; what is the smallest step you can take?

Renaming a variable? Moving a method? Extracting an interface? Pulling a field?...

Practices

What do you need to do to perform a valid refactoring? I’ll show you some practices.

The one I will always place forward first, is Automated Tests. Unit Tests should have a virtual code coverage percentage of 100% at all time of your code. In XP and other methodologies, you also need to define Acceptance Tests to perform story building tests at system-level.

Both type of tests is written first in a Test-First scenario. Refactoring is the third other part that define Test-Driven-Development (TDD).

Test-First + Refactoring = Test-Driven-Development

You first write a failing unit test at the beginning of your implementation, and you first write a failing acceptance test in the beginning of your agile iteration. That’s ideally. Some write their tests after implementation in a Test-Least-Development approach; but I think that every respectable developer thinks about the tests while developing and whether you write first your tests and then the implementation or write directly after your implementation your tests: in both cases, you think of how your production code will be tested and that’s the purpose of a Test-First approach, you’re only doing it backwards in a Test-Least Development scenario.

So, I leave it up to you what approach you will use, but I feel more comfortable when I writing my tests first.

Test – Code – Refactor

Another practice that you will absolutely need is Continuous Integration. By continuously integrating with your team members, will decrease unpredictability and the cost of integration. Develop your features in a different branch, but always integrate in a couple of hours. It’s a bad practice if you can’t point out a valuable reason to have multiple code streams, you’ll have an enormous waste in your software.

Instead of having continuously multiple streams, fix your underlying design problem by refactoring.

How, Why, and When?

Now that I give you a quick introduction, I’ll show you some basic concepts around Continuous Refactoring.

How Should You Refactor?

The easiest one is the How. How should you refactor? That’s rather simple, Martin Fowler and Joshua Kerievsky has started the refactor pattern story by creating a starting point of what the mechanics are in a refactoring environment.

Martin talks about low-level refactorings like Move Method, Encapsulate Field, Replace Inheritance with Delegation, Decompose Conditional, Encapsulate Collection… while Joshua talks about composed and test-driven refactorings like Replace Implicit Tree with Composite, Unify Interfaces with Adapter, Encapsulate Composite with Builder

A lot of these refactoring patterns are based on a pattern Expand – Migrate – Contract which allows Parallel Change and backwards compatibility. You should always hold in mind that during every step, EVERYTHING MUST STILL WORK. The pattern of Expand – Migrate – Contract tells you to expand your code base always first instead of directly change your code. Because you first add new code instead of adapting existing; you create an environment of backwards-compatible components. Just like unit tests are given you confidence, so does this pattern.

As example I can talk about a “simple” refactoring pattern Add Parameter from Martin. This pattern describes how to add a parameter to a method. Instead of just adding the parameter to the existing method, you first create a new method with this added parameter (Expand phase). After that, you change every call to the old method to the new one (Migrate phase). And as last, you remove the old method so the new method is now fully integrated (Contract phase).

After each phase, you compile/test, but also after each change from method calls. The process described in here for Add Parameter is how you refactor. (NOTE: C#, Java, Python… has the possibility to add default-value parameters to methods, which would help you with this refactoring. But I find it always useful to know more than a single solution for a problem).

What’s important is that you take Baby Steps in your refactoring process and slowly go to, towards or away from a current implementation choice by testing after every transformation step.

Why Should You Refactor?

It bit harder one to explain (especially to non-technical people) is the Why. Why should you refactor? Try to explain to a customer or non-technical manager that you will change your software so it will have more quality, and it would be easier to maintain and expand without there’s anything that will be changed or added of the current implemented functionality.

“If you aren’t going to change or add anything, could we just add the next feature?”

Business people think in features. Features added, removed, changed… but never refactored. The very definition of refactoring is that you are not going to change any outside observable behavior.

There is a way, though, that you can use to explain what “refactoring” really means to those people. It calls Technical Debt (synonym for Code Debt or Design Debt). Technical (or code or design) debt describes the virtual debt you must repay if you don’t decrease your debt. This is often called the Financial Metaphor and is a valuable solution to explain to non-technical people what the cost of not-refactoring means.

The next time you must defend yourself against someone who doesn’t see the risk your taking by NOT refactoring, pull out your credit card and tell them: “Would you skip the payment of your bills every month?”. Just like your own financial bills that you must pay to get back on track, so must you spent time refactoring so you don’t get in a situation that you can’t pay back your Technical (Code/Design) Debt.

Why should we refactor? Because we just can’t have THE perfect solution in the begin of the project with Up-Front Design so we must refactor daily to get each time closer to THE solution.

There are many reasons why you should refactor: change/adding/removing a feature will become easier, people will understand more quickly your design, bugs will experience more difficulty to hide, your code will remain fresh (without smells), …

When Should You Refactor?

"Seeing the trouble spot is often the hardest and most uncertain part"
- Eric Evans, Domain-Driven Design

The When is probably the hardest one. It’s a very lame answer but experience is the long-way to gain knowledge of the when; and you only gain experience by learning from making right and wrong decisions.

Reading books about refactoring (I’ve learned a lot from Martin Fowler, Joshua Kerievsky, Robert C. Martin; a.k.a. Uncle Bob…), Continuously Refactor and; fail. That’s the way you will start to learn when to refactor. Most design problems come when your design is:

  • Duplicated, implicit or explicit
  • Unclear, don’t show codes intent
  • Too complicated, over-engineered, not the simplest thing that could work

To spot places that need refactoring, you can start with learning different types of Code Smells. These smells will indicate that something is strange about the design, but hold on. The smells are only indicators and are not always the problem itself but only signals that there may be a problem. 

Some smells from Martin that are introduced in his Refactoring book: Feature Envy, Shotgun Surgery, Primitive Obsession, Refused Bequest… By building a vocabulary of refactorings patterns and code smells, you can quicker communicate with your team members about problem indicators you’ve found and solutions you propose.

You can also start introduce a “Code Smell Day” (typically at the end of each iteration) by which each team members starts looking in the code base at places that Smell. Or you can introduce a Refactoring Day in which you only refactor all the left over TODO items that needs to be refactored so your debt is repaid.

That’s for the technical part; the other part is about planning and time management. If you spot a candidate for refactoring; I find it always hard to place it in the scale of Reporting/Not-Reporting.

When the refactoring is about the feature you’re implementing and it will benefit the further process; then there’s no doubt, than you MUST refactor. If on the other hand, you find some smells in other parts of the system and it has nothing to do with what you’re doing. It could be useful to have a Public Refactor List by hand to describe what you’ve found and discuss it at the Weekly Meeting of the iteration so it can be a part of the next iteration.

OR you can refactor it right away and tell everyone of the team what you’re doing. Continuous Integration plays a big role in this one.

What I was saying about the Reporting/Not-Reporting part, is about the scale of your refactoring. If you refactor a method by performing Rename Method on it, or removing a temporary variable by executing Inline Temp... it would be an overload to define a work item for each refactoring; it should be a part of your development work. Just like you don’t specify for each feature a task “Writing Unit Tests”, you should not specify items with this low-level refactorings (even composite or test-driven refactorings). It should be a part of your work. Period.

But if the refactoring is a chain of multiple low-level, composite, or test-driven refactorings; than it smells more like a Design Refactoring and could it be of use to add this to the global planning as a story.

I find it sometimes difficult to know when you should Report the refactoring or Not. Should it be a Planned Refactoring? Should it be a Long-Term Refactoring?

The more technical Refactoring Workflows are already part of the Development Workflow; so, it would be redundant to specify these in the planning.

Conclusion

Refactoring is nowadays widely adopted in teams as a natural thing; although there’s still so many misunderstanding about the topic and what risks you and your team takes by NOT Continuously Refactoring. It should be a part of your development process; just like writing Unit- and Acceptance- Tests before implementing.

Practice your Continuous Refactoring skills daily and don’t be afraid to change your code daily. It’s a lot riskier to leave a dirty piece of code and to make workarounds every day than to change it so everyone can feel more comfortable.

Always think you’re programming in clay and not in stone: with some water, you can always continuous change the structure of your clay; while water only destroys stone.

Refactoring removes Duplication, increases Readability, Simplicity, and the Code’s Intent

So, start refactoring your project continuously and without any mercy for any written piece of code. When this is your daily practice; you may call yourself a Merciless Refactorer.

Categories: Architecture
written by: Stijn Moreels

Posted on Wednesday, February 1, 2017 3:00 PM

Toon Vanhoutte by Toon Vanhoutte

In many integration projects, the importance of idempotent receivers is overlooked. This blog post summarizes what idempotent receivers are, why we need them and how we can achieve it.

What?

Let's first have a closer look at the definition of idempotence, according to Wikipedia. "Idempotence is the property of certain operations in mathematics and computer science, that can be applied multiple times without changing the result beyond the initial application." The meaning of this definition is explained as: "a function is idempotent if, whenever it is applied twice to any value, it gives the same result as if it were applied once; i.e., ƒ(ƒ(x)) ≡ ƒ(x)".

If we apply this on integration, it means that a system is idempotent when it can process a specific message multiple times, while still retaining the same end-result. As a real-life example, an ERP system is idempotent if only one sales order is created, even if the CreateSalesOrder command message was submitted multiple times by the integration layer.

Why?

Often, customers request the integration layer to perform duplicate detection, so the receiving systems should not be idempotent. This statement is only partially true. Duplicate detection on the middleware layer can discard messages that are received more than once. However, even in case a message is only received once by the middleware, it may still end-up multiple times in the receiving system. Below you can find two examples of such edge cases.

Web service communication

Nowadays, integration leverages more and more the power of API's. API's are built on top of the HTTP protocol, which can cause issues due to its nature. Let's consider the following situations:

  1. In this case, all is fine. The service processed the request successfully and the client is aware of this.

  2. Here there is also no problem. The service failed processing the request and the client knows about it. The client will retry. Eventually the service will only process the message once.

  3. This is a dangerous situation in which client and service are misaligned on the status. The service successfully processed the message, however the HTTP 200 response never reached the client. The client times out and will retry. In this case the message is processed twice by the server, so idempotence might be needed.

Asynchronous queueing

In case a message queueing system is used, idempotency is required if the queue supports guaranteed at-least-once delivery. Let's take Azure Service Bus queues as an example. Service Bus queues support the PeekLock mode. When you peek a message from the queue, it becomes invisible for other receivers during a specific time window. You can explicitly remove the message from the queue, by executing a Complete command.

In the example below, the client peaks the message from the queue and sends it to the service. Server side processing goes fine and the client receives the confirmation from the service. However, the client is not able to complete the message because of an application crash or a network interference. In this case, the message will become visible again on the queue and will be presented a second time towards the service. As a consequence, idempotence might be required.

How?

The above scenarios showcase that duplicate data entries can be avoided most of the time, however in specific edge cases a message might be processed twice. Within the business context of your project, you need to determine if this is an issue. If 1 out of 1000 emails is sent twice, this is probably not a problem. If, however 1 out of 1000 sales orders are created twice, this can have a huge business impact. The problem can be resolved by implementing exactly-once delivery or by introducing idempotent receivers.

Exactly-once delivery

The options to achieve exactly-once delivery on a protocol level are rather limited. Exactly-once delivery is very difficult to achieve between systems of different technologies. Attempts to provide an interoperable exactly-once protocol, such as SOAP WS-ReliableMessaging, ended up very complex and often not interoperable in practice. In case the integration remains within the same technology stack, some alternative protocols can be considered. On a Windows platform, Microsoft Distributed Transaction Coordinator can ensure exactly-once delivery (or maybe better exactly-once processing). The BizTalk Server SQL adapter and the NServiceBus MSMQ and SQL transport are examples that leverage this transactional message processing.

On the application level, the integration layer could be made responsible to check first against the service if the message was already processed. If this turns out to be true, the message can be discarded; otherwise the message must be delivered to the target system. Be aware that this results in chatty integrations, which may influence performance in case of a high message throughput.

Idempotent receiver

Idempotence can be established within the message itself. A classic example to illustrate this is a financial transaction. A non-idempotent message contains a command to increase your bank balance with € 100. If this message gets processed twice, it's positive for you, but the bank won't like it. It's better to create a command message that states that the resulting bank balance must be € 12100.  This example clearly solves idempotence, but is not built for concurrent transactions.

An idempotent message is not always an option. In such cases the receiving application must take responsibility to ensure idempotence. This can be done by maintaining a list of message id's that have been processed already. If a message arrives with an id that is already on the list, it gets discarded. When it's not possible to have a message id within the message, you can keep a list of processed hash values. Another way of achieving idempotence is to set unique constraints on the Id of the data entity. Instead of pro-actively checking if a message was already processed, you can just give it a try and handle the unique constraint exception.

Lately, I see more and more SaaS providers that publish idempotent upsert services, which I can only encourage! The term upsert means that the service itself can determine whether it needs to perform an insert or an update of the data entity. Preferably, this upsert is performed based on a functional id (e.g. customer number) and not based on an internal GUID, as otherwise you do not benefit from the performance gains.

Conclusion

For each integration you set up, it's important to think about idempotence. That's why Codit has it by default on its Integration Analysis Checklist. Investigate the probability of duplicate data entries, based on the used protocols. If your business case needs to avoid this at all time, check whether the integration layer takes responsibility on this matter or if the receiving system provides idempotent service endpoints. The latter is mostly the best performing choice.

Do you have other ways to deal with this? Do not hesitate to share your experience via the comments section!

Thanks for reading!

Toon

Categories: Architecture
Tags: Design
written by: Toon Vanhoutte

Posted on Monday, November 21, 2016 4:11 PM

Stijn Moreels by Stijn Moreels

What drives software design? Several mechanisms are introduced over the years that explains what different approaches you can use or combine. Following article talks about just another technique: Responsible-Driven Design, and how I use it in my daily development work.

1    INTRODUCTION

“You cannot escape the responsibility of tomorrow by evading it today”
-Abraham Lincoln

Have you taken responsibility in your work lately? I find it frustrating that some people may not take responsibility in their software design and just writing software until it “works”. We all need to take responsibility in our daily work; software design/development work is no exception.

What drives software design? Several mechanisms are introduced over the years that explains what different approaches you can use or combine. Test-Driven Development talks about writing tests first before writing production code, Data-Driven Development talks about defining processing strategies in function of your data, Domain-Driven Design talks about solving a domain problem by using the vocabulary of the Ubiquitous Language with a high-level abstraction, Behavior-Driven Design is in short extension of Test-Driven Development with concepts of Domain-Driven Design…

Following article talks about just another technique: Responsible-Driven Design, and how I use it in my daily development work.

Rebecca Wirfs-Brock (founder of RDD) talks about some constructs and stereotypes that defines this approach and helps you build a system that takes responsibility at first place. I’m not going to give you a full description about her technique, though I’m going to give you a very quick view of how this can be used to think about your daily designing moments; at high-level and low-level software design.

A lot of programmers don’t think that they should be involved in designing the software, “That’s work for the Application Architect”. Their so wrong. This article is for developers, but also project managers and architects.

"Software Development is all design"
—Eric Evans

Your goal should not be to find THE solution at the beginning of your design process; it should be your goal to have a flexible design that you can refactor constantly to have THE solution at the end of your design process.

“Good programmers know they rarely write good code the first time”
—Martin Fowler

"The problem with software projects isn't change, per se, because change is going to happen; the problem, rather, is the inability to cope with change when it comes."
—Kent Beck

2    DEFINING RESPONSIBILITIES

“Understanding responsibilities is key to good object-oriented design”
—Martin Fowler

Defining responsibilities is crucial to have a good software design. Use Cases are a good starting point for defining responsibilities. These cases state some information of "What if… Then… and How" chains. Though, it isn't the task of use cases to define coordination or control of the software or the design; these tasks you must define yourself.
 
Rebecca describes some Roles to describe different types of responsible implementations to help you define software elements. A “Role” is some collection of related tasks that can be bundled to a single responsibility.

  • Service providers: designed to do things
  • Interfaces: translate requests and convert from one level of abstraction to another
  • Information holders: designed to know things
  • Controllers: designed to direct activities
  • Coordinators: designed to delegate work
  • Structurers: manage object relations or organize large numbers of similar objects

Some basic principles of responsibilities are the following: doing, knowing and deciding. Some element does something, another knows something and another decides what's next or what must be done.
This can help in the defining process of responsibilities. Mixing more than one of these principles in a single element is not a good sign of your design.

When structuring the requirements, and use cases, try to find the work that must be done, the information that must be known, the coordination/control activities, possible solutions to structure these elements…

It’s a good practice to define these responsibilities in the Software Design Document for each significant element. Rebecca talks about CRC Cards which list all the possible information one element knows, what work it must do…

3    ASSIGNING RESPONSIBILITIES 

Data-Driven Design talks about a centralized controlled system in its approach to have application logic in one place; this get quick complex though. Responsible-Driven Design is all about delegated control to assign responsibilities to elements and therefore each element can be reused very quickly because it only fulfills its own responsibilities and not ones from some other elements in the design. Distributing to much responsibilities can lead to weaker objects and collaboration/communication of objects.

When assigning responsibilities, there’s a lot of Principles/Patterns that helps me to reflect constantly on my design. Here are some I think about daily:

Keep information in one place:     
“Single Point of Truth Principle”: principle that states that each piece of information is stored exactly once.

Keep a responsibility small:     
“Law of Demeter - Principle of the least knowledge”: each element should have a limited piece of knowledge stored itself and have about other elements (see also Information Hiding and Information Expert).

Wrap related operations:     
“Whole Value Object” (for example): wrap related operations/information in an object on its own and give it a descriptive name.

Only use what you need:    
“Interface Segregation Principle”: each interface should only implement method which it needs.

Aligned responsibility:    
“Single Responsible Principle (a class should only have one reason to change)”: each part of your software should have a single responsibility and this responsibility should entirely be wrapped in this part.
 
And so, so many more…

4    CLASS RESPONSIBILITIES

When defining classes, the Single Responsibility Principle comes in mind. Each class should do only one predefined task/responsibility. A trick I use to keep me always aware of this responsibility, is to write above each class in comments what that class should do and only do.
 
If you find yourself writing words like: "and", "or", "but", "except"… you're probably trying to do more than just one thing. It's also very useful when adding new code to check if it's still in the class its responsibility; if not, rethink your design to find the exact spot to where to put your new code. Also, try to speak in one sentence about responsibility.
 
Classes which names contains: "manager", "info", "process"… is also an indication that you're doing something more than it should in your class. It could also mean that you named the class with such a general name because otherwise it will not state the class its responsibility.

In that way, you have a "nice" Anti-Pattern in place which I like to call: Responsibility-Hiding Anti-Pattern. Hiding of responsibilities should (of course) be eliminated, it only obscures the design and is a weak excuse for a bad design. A class named “ElementProcessor” for example is a perfect example; “What’s happening in the Process-part?”. It feels the class has some black magic in it and you can call the class to say: “Do the magic!”. Use strong descriptions when defining responsibilities for your class, if you can't do that, refactor your class until you can.

One of the best reasons to create classes is Information Hiding. If a subset of methods in a class uses a subset of information. Then that part could/must be refactored into a new class. This way we have not only Hide the information, we have refactored the data and logic so it’s on the same Rate of Change.

Can you spot following misplaced responsibility?
 


Is it the job of the File Attachment Uploader class to know what extension the Attachment should need? What if I need a FTP Attachment Uploader?

For people who wants more read: assigning to many responsibilities is actually related to the Anti-Patterns from Brown: Software Development Anti-Pattern: Blob/God-Class Anti-Pattern which talks about a single class that contains a bunch information, has some logic exposed… a class that contains SO MANY RESPONSIBILITIES; and the Software Architecture Anti-Pattern: Swiss Army Knife Anti-Pattern which talks about a complex interface (so multiple implementations) that has multiple RESPONSIBILITIES and is used in to many software problems (as a Swiss Army Knife) as solution.

5    FUNCTION RESPONSIBILITIES

I told you that I get very deep, now I'm going to talk about just a function. What of responsibility has a function for example. The name of a function should always be a verb for a start. Something to do.

Naming is key in good communication; which follows also another Principle: Principle of Least Surprise. Only to look at some name you should have a clue what’s been done in that function/class/package… so you don’t get surprised.

A function should (definitely) only do one thing. Robert C. Martin talks about different ways to spot if a function does more than one thing: if you can extract a part of the function and give a meaningful name that doesn’t restate the original function name; you’re doing more than one thing; and so, have multiple responsibilities.

 Looking back at the previous example (after refactoring of the “GetExtension” method). Does “Upload Attachment” one thing? No, it gets first the exact path from attachment related information.


Of course, this example and still be refactored, and please do. Also, note that the extracted function called “GetAttachmentLocation” and not “GetLocation”. Because we added the “Attachment” part. The function logically gets an attachment as argument from the “UploadAttachment” function.

Try to always use descriptive names in your functions and say what you do in the function, not HOW you do it. Name your function after its responsibility. If we named the function “GetLocation” there would be not logically explanation why we would send an Attachment with it because it isn’t in the function its responsibility.


After again a refactoring session, we could see that there’s maybe a missing concept. Why doesn’t have the Attachment a location as information? Also, I don’t like to see if a function has one or multiple parameters AND a return value. It violates the Query-Command Separation Principle.


Note that we also extracted the function with the File Stream related information. We did this so each function is on the same level of abstraction. Each function should do as the name says it does, no surprises. And every time you go to the next implementation, you should get deeper, more concrete and less abstract. This way each function exists on one and only one layer of abstraction. “UploadAttachment” is on a higher layer of abstraction that “AssignAttachmentLocation” and “SaveAttachmentToFileSystem”.

It’s the responsibility of the “UploadAttachment” function to delegate to the two functions and not to try do something concrete itself.

Now, we could still refactor this because there isn’t any exception handling functionality for example. Just like refactoring is an iterative process, so is software designing. Please constantly refactor your code to a better approach.

I rename my classes/functions/… daily for example. I then look at it from a distance and think: “does this really explains what I want to say?”, “Is there any better approach?”, “Is it the responsibility of this variable to know this?”, “Does this function only its responsibility?” …

6    DOCUMENTATION RESPONSIBILITIES

One of the reasons documentation exist, is to define responsibilities. Many developers don’t see the purpose of defining a software design documentation, because it's extra work and gets out of date fast. But also, because it explains something they already know (or think they know). In this document, you describe what each element/package's responsibility is, why you named something like that…
 
Most of the time, I define for each layer in my design (see Layered Architecture) a separated block of documentation. Each title starts with the same explanation: Purpose. What's the reason this package exists? After that a Description section is placed to describe some common information and responsibility the layer has. Next, is the UML Schema/Interaction Diagram section where schemas are placed to have some technical description how object are connected (UML) and collaborate (Interaction) with each other.

Eric Evans state that we should name our layers not only by their technical term like: Infrastructure Layer, Domain Layer, Application Layer, Presentation Layer… but also by their Domain Name. What Role plays this layer in your software design?

So, to summarize:

  • Title
  • Purpose
  • Description
  • UML Schema
  • Interaction Diagram

Just like writing Tests make you think about dependencies and help you to rework your design to a better approach; helps writing high-level software documentation me to get the purpose of each layer in the design. While typing purposes/descriptions/responsibilities… in my documentation, I somethings stops and thinks: “Didn’t I just wrote a piece of code that doesn’t fall in this layer responsibility?”.

Interaction Diagrams are less popular than UML but describes the actual flow your design is following. You quick find a spot in your diagrams where there are just too many arrows. This also helps you think about coordination and control of your design. Please do not underestimate this kind of approach, it helped me to get the design in its whole form.

If your team plans a weekly code freeze, this could be an ideally time to update the documentation and schema’s. This not only keeps track of your changes and responsibilities, it also helps to introduce the design to new members of the team.

This kind of approach helps me to move elements through layers to find a right home.

7    CONCLUSION

When writing every word of code, think about what you're doing. It is the right place to put this here? Is it my job to know that kind of information? Do I have the responsibility to do this? Why does this element have to make that decision? …

Every word, every line, every class, every package… has a responsibility and a purpose to exist. If you can't say way in a strong description why this class knows something, why a function has an argument, why a package is named that way… that you should probably put on your refactoring-head.
 
@Codit we aren’t satisfied just because our code works. The first obvious step in programming is that your code works; only then the real work begins…

Think about what you're doing and take responsibility.

Categories: Architecture
Tags: Design
written by: Stijn Moreels

Posted on Friday, July 15, 2016 1:26 PM

Tom Kerkhove by Tom Kerkhove

Today I will talk about how we are currently using code reviews to build better solutions and how it gives me more confidence in the quality that we are shipping.

Working in teams brings several challenges, one of them is a mixture of coding styles, causing code inconsistency across your project which makes it hard to read or follow what it does.

The bigger the team, the more important it is to transfer your knowledge about what you've worked on, so you have a limited Bus-factor in your team.

Or have you ever been working on a new feature fully confident that it's ready to ship, only to notice that you've forgotten to take into account about the caching? Or that you've forgotten to update the documentation (if any)?

Sounds familiar to you?

Note - While some of these "pain points" can be tackled by using tools like Roslyn Analyzers or FxCop I prefer a more humane approach and discuss the why instead of the how.

Code reviews to the rescue

By using code reviews we can avoid these problems by collaborating before we ship the code - Let's first take a look at an example:

While this code could perfectly process your order, there are some issues:

  • Variable o represents a certain amount of state but what does it represent? Looking at the signature it is clearly an Order, but how do I know that in a 50-line method at the bottom?
  • Time zones, they are evil! What happens if this code runs in the U.S.?
  • Calling MyMethod<T> takes in a boolean but what does it really do and how does the boolean come in to play?
  • How does the caller know what the Process method does? Hopefully bill for the order? Also, it couldn't hurt to add additional documentation throughout the implementation in certain cases.

While performing a code review, the reviewer can indicate these concerns with the reviewee and have a polite & constructive discussion, backed by a set of coding guidelines. By doing this, both parties get to know how the other person thinks about it and they learn from each other. Yet, they also learn to express how they did something or what they have forgotten about.

Having a second pair of eyes on a certain topic can help a lot. Everybody has a different perspective and this can make sure that you forget about a certain topic and also, potentially, leads to interesting discussions. This gives you a certain "Don't worry, I've got your back" feeling and forces you to think deeper about what you've written.

At the end of the review, the reviewee can have some feedback to process, where after the code gets the seal of approval and is ready to ship.

Next to the code quality, you also perform small knowledge transfers with each other. You will not remember everything but when needed you will remember certain pieces that can help you guide to the potential bug or cause.

Last but not least is automated testing. It's a good thing to add the unit/behavior/scenario testing to your reviews as well because then the reviewer gets an indication of what you are testing and what you are NOT testing. Do the tests make sense or should the reviewee cover additional scenarios?

Challenges

Using code reviews is of course not a free lunch and it comes with its own difficulties.

The biggest challenge is that your team members need to be open for feedback and willing to incorporate the feedback! If they are not up for it, you will just spend your valuable time to only notice that they are ignoring it. You, as a team, will need to decide whether or not you want to commit to the code-review-system.

Every review takes a decent amount of time, so incorporate that into your planning. The reviewer needs to go through it, discuss it with the reviewee and then the reviewee need to process the feedback. However, one might argue that it is better to take your time during development instead of having to spend twice that amount while fixing bugs or trying to understand what's going on.

New to code reviews? Here are some tips!

After using code reviews for a while, I've learned a couple of things on how to not do it or what can be challenging. Here are some tips that help you avoid some pitfalls.

Review early & frequently - The earlier you review, the better. This avoids reviewing something that is considered ready while you've misunderstood some aspects of it or have re-invented the wheel.

Define code guidelines - Agree upon a list of coding guidelines with your team to back your reviews. By doing this you have a clear list of styles, paradigms and DO's & DON'T DO's that the team should follow to unify your coding styles & practices. This makes reviewing a lot easier and a clear guidance on how it should look.

An example of this could be that each parameter should be checked for null and that it should throw an ArgumentNullException when appropriate.

Add code reviews to your definition-of-done - By adding code reviews to your definition of done you are certain that each new feature or bug fix has passed at least two pair of eyes and that multiple people agree on it.

By doing that, you also remove the burden of one person being responsible for one aspect, since it's the whole team that is responsible for it.

Don't review to bash, review to teach & improve - Finding a balance between being strict or agreeing with everything is hard. If you just bash it will have a negative impact on the team collaboration and frustrations will arise. Be constructive & open.

Review in-person but have a look at the changes in advance - This allows you to have a personal opinion instead of simply following the reviewee. This avoids you having to make decisions on the spot but instead digesting it first so you can think of obvious aspects

Challenge and be challenged - Ask questions about the topic to see if the reviewee has covered all the possible scenarios and learn about how they envision it. Discussions are a good thing, not a bad thing.

Learn from each other - Don't be afraid to say what you like and don't like or if you don't know about something. Learn from others and how he/she did it this way and not like how you thought about it.

Conclusion

While the internet has a wide variety of blogs talking about this "concept" I wanted to share my vision on this since I'm a big fan of this practice and believe that this really improves the quality. However, your success will depend on the cooperation of your colleagues and project management if they want to commit in doing so.

One thing is certain - I've used this on my current project, but will keep on doing so in the future.

Thanks for reading,

Tom.

Categories: Architecture
Tags: ALM
written by: Tom Kerkhove

Posted on Monday, March 21, 2016 11:48 AM

Luis Delgado by Luis Delgado

At Codit, we help customers envision, design and implement solutions focused on Internet of Things (IoT) initiatives. As part of this work, I've realized that organizations investing in IoT initiatives typically walk through a path, which I will call the "IoT Maturity Levels".

 

Maturity levels are important because they provide organizations with a goal-oriented path, so they can measure progress and celebrate small successes on their way to a greater goal. They are important because, as experience shows, it is best for organizations to progress through consecutive maturity levels rather than to simply try to swallow a super-complex project at once. Violent and ambitious jumps in maturity typically fail due to organizational change resistance, immature operational procedures, and deficient governance practices. It is better to have a solid hand on a maturity level before adventuring into the next one.

Here at the 4 maturity levels of IoT, and what they mean for organizations:

Level 1: Data Generation and Ingestion

What is it about: In level 1, organizations begin projects to generate and collect IoT data. This involves coupling their services or products with devices that capture data and gateways to transmit that data, implementing data ingestion pipelines to absorb that data, and storing that data for later use. 

What it means: at this point, companies are finally in a position to generate data and collect it. Data generation is the key building block of IoT, and the first maturity level is aimed at getting your hands on data. Typically, the hardest part are the devices themselves, how to securely capture and transmit the data, how to manage those devices in the field, solve connectivity issues, and building a pipeline that scales to serve many devices.

Level 2: First Analytics

What is it about: once armed with data, companies will typically try to derive some value out of it. These are initially ad-hoc, exploratory efforts. Some companies might already have a developed concept about how they will use the data, while others will need to embark in exploring the data to find useful surprises. For example, data analysts / scientists will start connecting to the data with mainstream tools like Excel and PowerBI and start exploring.

What it means: Companies might be able to start extracting value from the data generated. This will mostly be manual efforts done by functional experts or data analysts. At this stage, the organization starts to derive initial value from the data.

Level 3: Deep Learning

What is it about: the organization recognizes that the data is much more valuable and large than manual analysis permits, and starts investing in technology that can automatically extract insights from the data. These are typically investments in deep learning, machine learning or streaming analytics. Whereas the value of the data in Level 2 was extracted from the manual work of highly-skilled experts, the value of the data in Level 3 is extracted automatically from sophisticated algorithms, statiscal models and stochastic process modeling.

What it means: the organization is able to scale the value of its data, as it is not dependent anymore on the manual work of data analysts. More data can be analyzed in many more different ways, in less time. The insights gained might be more profound, due to the sophistication of the analysis, which can be applied to gigantic data sets with ease.

Level 4: Autonomous Decision Making

What is it about: the deep learning and analytical models, along with their accuracy and reliability, are solid upon exiting Level 3. The organization is now in a position to trust these models to make automated decisions. In Level 3, the insights derived from deep learning are mostly used as input for pattern analysis, reporting dashboards and management decision-making. In Level 4, the output of deep learning is used to trigger autonomous operational actions.

What it means: in Level 4, the deep learning engine of the organization is integrated with its operational systems. The deep learning engine will trigger actions in the ERP (i.e: automatic orders to replenish inventory), LoB systems (remote control of field devices via intelligent bi-directional communication), the CRM (triggering personalized sales and marketing actions based on individual customer behavior), or any other system that interacts with customers, suppliers or internal staff. These actions will require no human intervention, or at least, require minimal human supervision or approvals to be executed.

Do you need to go all the way up to Level 4?
Not necessarily. How far you need to invest into the maturity of your IoT stack is dependent on the business case for such an investment. The true impact of IoT, and what business value it might bring, is very hard to gauge at Day 0. It is best to start with smaller steps by developing innovative business models, rapidly prototyping them, and making smaller investments to explore if an IoT-powered business model is viable or not. Make larger commitments only as steps from previous successes. This allows you to fail fast with minimal pain if your proposed business model turns out to be wrong, adapt the model as you learn through iterations, and allows your team to celebrate smaller successes on your IoT journey.

Categories: Architecture
Tags: IoT
written by: Luis Delgado