Discover more from System Design Newsletter
Tech Stack Evolution at Levels.fyi
#8: Read Now - How Levels.fyi Went From No Backend to Serving 2.5 Million Unique Users a Month (5 Minutes)
Get the powerful template to approach system design for FREE on newsletter sign-up:
Dushyant from Levels.fyi reviewed this post before publishing.
Levels.fyi lets you compare career levels and compensation packages across different companies. It started as a side project. And now serves 2.5 million unique users a month.
This post outlines the tech stack evolution and startup lessons from Levels.fyi. Consider sharing this post with somebody who works at a startup.
Levels.fyi Tech Stack Evolution
Their tech stack went from no backend to a proper backend. Here is an overview of their tech stack evolution. V stands for Version.
Levels.fyi V0 🟩
They used Google Forms and Google Sheets. Google Forms collected data. While Google Sheets stored data.
They relied on Sheets API to query data. And backed up Google Sheets for durability.
Their reasons for using Google Forms and Google Sheets were:
Avoid having to build their own user interface only to collect data.
Avoid extra effort in processing and maintaining data.
Avoid extra tools to analyze, clean, and validate data.
Avoid the operational and maintenance effort.
Cut the need for data schema management, data migration, and ETL.
Familiarity with Google Sheets.
Easy to use.
In-built support for data access control.
Levels.fyi V1 🟩🟨
They built a web UI to serve read requests. And used lambda functions to read data from Sheets and store it in S3 as JSON files.
Yet this architecture had drawbacks:
Lack of support for SQL-based data analytics. This limited their ability to make data-driven decisions.
Hit the rate limiter threshold of Sheets API.
Lambda functions time out on processing large amounts of data.
Levels.fyi V2 🟩🟨🟧
They wrote the core backend services in Node.js (Nestjs). And used a static site generator.
The graphs or statistics shown on their website were not processed on the server side. But offloaded to the browser. This allowed scaling with minimal resources.
Also they built the API server using API Gateway and EC2. It helped to reduce latency and improve performance compared to V1.
They used the backend service for:
Preventing duplicate values.
And lambda functions for aggregating data and analytics.
Levels.fyi V3 🟩🟨🟧🟥
They moved the salary data out of Google Sheets. And built the infrastructure on the AWS stack.
AWS RDS (Postgres) stored critical data. While Metabase generated charts and visuals for internal use by analysts. The lambda functions aggregated the data (for example, percentile) by querying RDS.
They run Node.js (TypeScript) on the server. And orchestrate 5 microservices using a service mesh. The database gets replicated across many regions.
And use TinyStacks for continuous deployments.
But Google Sheets is still used in production for other use cases. There is a 10 million cell limit on Sheets. So, they sharded the Sheets as a workaround. Creating many folders and workspaces allowed sharding.
Here are the important lessons from this case study:
1. Keep It Simple
They wanted to focus on what's important - the product idea. So, they used no-code tools and avoided buzzwords. This allowed faster iteration.
2. Avoid Premature Optimization
They wanted to test their product idea. But not invest lots of time and money in doing it.
They used the AWS free tier with credits for the first 2 years. And moved to the paid tier after taking off.
3. Nothing Is Sacred
They outsourced their development and operational burden to Google and AWS. And kept changing the tech stack to meet the needs.
Consider subscribing to get simplified case studies delivered straight to your inbox:
A huge thank you to everybody who supports this newsletter. You are now 5,723 subscribers strong, very close to 6K.
I think we can get to 6,000 subscribers by September 30. Y’all are the best.
Disclaimer: I put a referral link to his newsletter. Because I’d love free access to content.
Last Week: 🚀 Catch up on the latest posts!
Word-of-mouth referrals like yours help this community grow - Thank you.
Dushyant Sabharwal (February 2023). How Levels.fyi scaled to millions of users with Google Sheets as a backend. [online] www.levels.fyi. Available at: https://www.levels.fyi/blog/scaling-to-millions-with-google-sheets.html [Accessed 19 Sep. 2023].
www.linkedin.com. (n.d.). Dushyant S. on LinkedIn: #engineering #startups | 16 comments. [online] Available at: https://www.linkedin.com/posts/dushyantsabharwal_engineering-startups-activity-6978282092782616576-VAIT/ [Accessed 19 Sep. 2023].
www.linkedin.com. (n.d.). We hit 10M cell limit on Google Sheets. Thanks Tanishq Singh! | Zuhayeer Musa posted on the topic | LinkedIn. [online] Available at: https://www.linkedin.com/posts/zuhayeer_googlesheets-salaries-salarytransparency-activity-7102715304924905472-fFH0 [Accessed 19 Sep. 2023].
www.youtube.com. (n.d.). Building a new website using Google Sheets and Google Forms. [online] Available at YouTube [Accessed 19 Sep. 2023].
www.youtube.com. (n.d.). How Levels.fyi Scaled to Millions of Users with Google Sheets. [online] Available at YouTube [Accessed 19 Sep. 2023].