The Game
Scalability is the ability to overcome performance limits by adding computational resources. In order to be able to add resources, your software architecture and design needs to support that scaling-up.
For example, Shopify served 1.27 Million requests per second during Black Friday. Their Kafka Streaming infrastructure processed 20 Million messages per second at peak. And still, Shopify’s main monolith is one of the oldest, largest Rails codebases on the planet. It is under continuous development since at least 2006, with over 2.8 million lines of Ruby code and 500,000 commits.
Nevertheless, the Shopify engineering team decided to deconstruct the monolith. They had outgrown the monolithic structure and something needed to be done.
What did the Shopify team get right? They successfully fulfilled their business mission with the right level of scale for a long time. Only when they really hit a performance need, they decided that scaling up is the right thing to do. Not before.
Seeing this, software scalability isn’t just a fun engineering exercise. It’s a game where businesses need to level up when demand justifies more scale. And if you can’t figure out how to play the game, you’ll probably never succeed to move your architecture to the next level.
The game is played in levels. And at each level, you acquire new skills and new points of leverage that you can use to move ahead.
Let’s pull back the curtain see how the game is played.
Level 1: The monolith
A single deployable application.
Even a single monolithic instance of, say nodejs, can already handle up to hundreds requests per second.
Set a load balancer in front of a handful instances, and you should be able to serve thousands of requests per second.
For most small and medium businesses, this type of application should be already good enough. A starting company should not even consider other forms of architecture unless its customer base grows.
The shopify example demonstrated that you can even handle over a million of requests per second with a monolith. However, you likely have reached the moment where you cannot scale anymore.
As a rule of thumb, you should almost always start with a monolith. And therefore, the first “level up” advice is: DO NOT level up unless you really need it!
Level 2: The modulith
A single deployable application composed of loosely coupled modules.
Modulithic architectures are easy to deploy, yet better to maintain than a monolith. For big companies or smaller enterprise applications, a modulith is the way to go.
How to level 🆙: Break up the monolith into separate domain modules which do not depend on each other. Safeguard that boundaries without making exceptions with Gradle, the archunit library or others. Have a look at Spring Modulith, if you are a Spring user. Communicate between the modules with an internal, synchronous adapter middleware. If you really need asynchronous communication you can use Spring domain events or another event-based architecture.
The main point is to only couple loosely the modules because a well-designed modulith represents already the blueprint of a small microservice architecture. It is not premature optimization to architect your modulith in a way that it can be scaled up to a microservices architecture. Great architecture is easy to extend and makes scaling up easy.
Level 3: Microservices
Microservices consists of multiple deployable software assets that scale independently according to their need. A microservice should do “one thing right” (which refers to a domain problem!) and may come with a separate data storage.
While you can achieve superior scalability with microservices, orchestration comes at the cost of higher complexity.
How to level 🆙: Before even thinking about a Microservices architecture, make sure you have reached level 2. You should have established clear model of domain boundaries. It is much easier to break up a well-defined modulith than a messy, entangled monolith. You should really hit a wall with your modulithic scalability constraints before escalating to the next step.
Again, the modulith architecture is the blueprint of the microservices architecture. It is easier to scale out to microservices if the structures are well established.
Bonus Level 4: Cloud native planet scale
A few huge corporates, notably, Big Tech (FAANG) and other planet scale companies have reached a level where “everything scales”.
Are you FAANG? Probably not. Therefore, you often do not need to achieve this level of scale.
How to level 🆙: Make sure your application landscape is cloud-native. Every asset in the architecture should be scalable. As the architect you probably have to shift to a data-driven approach to observe your architecture.
Take away
Congratulations! You have mastered the scalability game 😄
From now on it should be obvious that the choice of scale depends on your business needs. If you work in FAANG “will it scale?” is a legitimate question. If you work in a startup without customers yet, maybe it’s premature.
That being said, optimize for adaptability rather than scalability. It is much easier to level up understandable code scalable than the other way around.
