Scalability within micro-services architecture

Vertical scalability

With vertical scalability, you scale the capacity of your application by increasing hardware capacity. This is why it is named “vertical”: you add more CPU, more memory, more disk IOPS, but effectively, the architecture of your app and infrastructure does not change. This is a viable scalability pattern but it has an obvious hard-stop: there is so much hardware you can add, up to a certain point.

Horizontal scalability

With horizontal scalability, instead of scaling up by adding more hardware capacity, you architect your application so that it scales out by adding more instances of it. This can be accomplished by adding more VMs with the application installed, more application instances inside a cloud service, or more containers… you get the idea. You don’t need beefy, expensive hardware for horizontal scalability, you can get along with small machines, and add many of them. This scalability pattern usually requires adjustments in the application architecture. For example, since a client request may be served by any machine (out of many), the application typically has to be stateless, or if state is needed, it needs to be stored somewhere else.

Scalability with micro-services

Whereas the concept of scaling horizontal seems appealing, remember that every instance you “clone” to scale horizontally is running a complete instance of your application. This might be undesirable, as your application might have different scalability needs. Typically, the load of an application is not evenly-distributed among all the services it provides. For example, a careful analysis of telemetry data might show that the bottleneck in your application are the authentication services, but all other services inside your app are performing well. If you scale horizontally, you will be scaling out the authentication services… along with everything else that does not need to be scaled out. This is a waste of resources.

A micro-services architecture takes an application and splits it into independent, working, functional units, called “services”. Don’t get mislead by the word “micro” in “micro-services”. The split of services does not need to be “microscopic”. You can split the services within your application in any arbitrary way you want. Typically, the more atomic the services are, the more value you will get from this architecture, but it need not be the case every time.

With that out of the way, let’s go back to our example. With a micro-services architecture, your app will run different services as independent units, each with its own runtime, codebase, processing thread(s), etc. Since the bottleneck in our app is the authentication routine, you can scale out that service only, and leave the rest alone. With micro-services, you make a more effective use of the horizontal scalability pattern.

When considering a micro-services architecture, there are many more factors to analyse beyond scalability. But mixing the micro-services architecture with horizontal scalability typically gives you better capacity elasticity than using a monolithic architecture.