How Shopify Handled 30TB per Minute With a Modular Monolith Architecture 🔥
#63: Break Into Modular Monolith Architecture (3 Minutes)
Get my system design playbook for FREE on newsletter signup:
This post outlines modular monolith architecture. You will find references at the bottom of this page if you want to go deeper.
Share this post & I'll send you some rewards for the referrals.
Note: This post is based on my research and may differ from real-world implementation.
Once upon a time, 3 people decided to sell snowboarding equipment online.
And set up an e-commerce platform.
Yet their business model failed.
So they pivoted to create a software-as-a-service model with the e-commerce platform and called it Shopify.
Their growth rate was explosive.
But they wanted to keep it simple and reduce costs.
So they built Shopify using a classic monolith architecture in Ruby on Rails.
As time passed, they added features to scale the business.
Yet it increased the codebase complexity.
And became difficult to understand & maintain the code.
So they considered moving to a microservices architecture.
Although it’ll temporarily solve their issues, it might create newer problems.
Here are some of them:
1. Infrastructure Costs
Each microservice needs a separate deployment pipeline.
Besides service discovery and API gateway should be set up.
So it’ll increase the effort and costs.
2. System Reliability
There’s a risk of failure from an unreliable network.
Also security vulnerabilities must be updated in each microservice.
So it’ll increase the system complexity.
Onward.
Modular Monolith
They wanted the benefits of monolith and microservices architecture.
But ditch the problems coming with it.
So they set up a modular monolith.
Here’s how:
1. Design Principles
They designed each module around a specific business domain.
It gave high cohesion.
Think of cohesion as grouping functionalities from a business area within a module. Put simply, related functionalities get stored in a specific module.
Also they created a public interface for each module and hid the internal logic & data behind it. It let them:
Achieve low coupling.
Understand the behavior of a module easily.
Modify a module without breaking others.
Imagine coupling as a measure of dependency between 2 modules.
A module will always depend on something.
Yet dependencies must be minimized.
So they merged modules with many dependencies on each other. (It happens when domain boundaries aren’t set properly.)
They designed modules to communicate with each other via public API - function calls. And threw errors if anything got accessed outside the public API.
Each module can be deployed as a separate process for fault isolation.
Yet it’ll increase data serialization costs and network overhead.
So they deploy every module within the same process.
And the best part?
2. Code Organization
They organized code so each module contains a business domain. Put simply, each directory represents a specific business domain. For example, the code for orders and payments modules will be in separate directories.
Also they set up separate database schemas for each module. And avoided transactions across them to avoid coupling & achieve data isolation.
3. Benefits
A modular monolith is a better way to organize code than a classic monolith. Here are its benefits:
It’s easier to modify code as the area of code change is limited. (Compared to layered architecture in a classic monolith.)
Developer productivity is high because the codebase is easier to understand.
There’s no network overhead because functional calls are enough. (Compared to microservices.)
It’s easier to deploy because there’s only a single app.
The infrastructure costs are low because there’s only a single codebase.
Besides a modular monolith makes the transition to microservices easier.