Tech Stack Evolution at Levels.fyi
Discover more from System Design Newsletter
Tech Stack Evolution at Levels.fyi
#8: Read Now - How Levels.fyi Went From No Backend to Serving 2.5 Million Unique Users a Month (5 Minutes)
Get the powerful template to approach system design for FREE on newsletter sign-up:
Dushyant from Levels.fyi reviewed this post before publishing.
Levels.fyi lets you compare career levels and compensation packages across different companies. It started as a side project. And now serves 2.5 million unique users a month.
This post outlines the tech stack evolution and startup lessons from Levels.fyi. Consider sharing this post with somebody who works at a startup.
Levels.fyi Tech Stack Evolution
Their tech stack went from no backend to a proper backend. Here is an overview of their tech stack evolution. V stands for Version.
Levels.fyi V0 🟩
They used Google Forms and Google Sheets. Google Forms collected data. While Google Sheets stored data.
They relied on Sheets API to query data. And backed up Google Sheets for durability.
Their reasons for using Google Forms and Google Sheets were:
Avoid having to build their own user interface only to collect data.
Avoid extra effort in processing and maintaining data.
Avoid extra tools to analyze, clean, and validate data.
Avoid the operational and maintenance effort.
Cut the need for data schema management, data migration, and ETL.
Familiarity with Google Sheets.
Easy to use.
Save money.
In-built support for data access control.
Levels.fyi V1 🟩🟨
They built a web UI to serve read requests. And used lambda functions to read data from Sheets and store it in S3 as JSON files.
There was no front-end framework used. Instead plain JavaScript, jQuery, CSS, and HTML.
Yet this architecture had drawbacks:
Lack of support for SQL-based data analytics. This limited their ability to make data-driven decisions.
Hit the rate limiter threshold of Sheets API.
Lambda functions time out on processing large amounts of data.
Levels.fyi V2 🟩🟨🟧
They wrote the core backend services in Node.js (Nestjs). And used a static site generator.
The graphs or statistics shown on their website were not processed on the server side. But offloaded to the browser. This allowed scaling with minimal resources.
Also they built the API server using API Gateway and EC2. It helped to reduce latency and improve performance compared to V1.
They used the backend service for:
Spam detection.
Rate limiting.
Preventing duplicate values.
And lambda functions for aggregating data and analytics.
Levels.fyi V3 🟩🟨🟧🟥
They moved the salary data out of Google Sheets. And built the infrastructure on the AWS stack.
AWS RDS (Postgres) stored critical data. While Metabase generated charts and visuals for internal use by analysts. The lambda functions aggregated the data (for example, percentile) by querying RDS.
They run Node.js (TypeScript) on the server. And orchestrate 5 microservices using a service mesh. The database gets replicated across many regions.
And use TinyStacks for continuous deployments.
But Google Sheets is still used in production for other use cases. There is a 10 million cell limit on Sheets. So, they sharded the Sheets as a workaround. Creating many folders and workspaces allowed sharding.
Takeaways
Here are the important lessons from this case study:
1. Keep It Simple
They wanted to focus on what's important - the product idea. So, they used no-code tools and avoided buzzwords. This allowed faster iteration.
2. Avoid Premature Optimization
They wanted to test their product idea. But not invest lots of time and money in doing it.
They used the AWS free tier with credits for the first 2 years. And moved to the paid tier after taking off.
3. Nothing Is Sacred
They outsourced their development and operational burden to Google and AWS. And kept changing the tech stack to meet the needs.
Consider subscribing to get simplified case studies delivered straight to your inbox:
Readership
A huge thank you to everybody who supports this newsletter. You are now 5,723 subscribers strong, very close to 6K.
I think we can get to 6,000 subscribers by September 30. Y’all are the best.
Further Read
Did you know that is on Substack? He writes the Software Design: Tidy First? Newsletter. Also he is publishing a book titled Tidy First? next month.
Disclaimer: I put a referral link to his newsletter. Because I’d love free access to content.
Read this post summary: Twitter Thread, Substack Note, and LinkedIn Post.
Last Week: 🚀 Catch up on the latest posts!
Reader Praise
Word-of-mouth referrals like yours help this community grow - Thank you.
Get featured in the newsletter: Write your feedback on this post. And tag me on Twitter, LinkedIn, and Substack Notes. Or, you can reply to this email with anonymous feedback.
References
Dushyant Sabharwal (February 2023). How Levels.fyi scaled to millions of users with Google Sheets as a backend. [online] www.levels.fyi. Available at: https://www.levels.fyi/blog/scaling-to-millions-with-google-sheets.html [Accessed 19 Sep. 2023].
www.linkedin.com. (n.d.). Dushyant S. on LinkedIn: #engineering #startups | 16 comments. [online] Available at: https://www.linkedin.com/posts/dushyantsabharwal_engineering-startups-activity-6978282092782616576-VAIT/ [Accessed 19 Sep. 2023].
www.linkedin.com. (n.d.). We hit 10M cell limit on Google Sheets. Thanks Tanishq Singh! | Zuhayeer Musa posted on the topic | LinkedIn. [online] Available at: https://www.linkedin.com/posts/zuhayeer_googlesheets-salaries-salarytransparency-activity-7102715304924905472-fFH0 [Accessed 19 Sep. 2023].
www.youtube.com. (n.d.). Building a new website using Google Sheets and Google Forms. [online] Available at YouTube [Accessed 19 Sep. 2023].
www.youtube.com. (n.d.). How Levels.fyi Scaled to Millions of Users with Google Sheets. [online] Available at YouTube [Accessed 19 Sep. 2023].
Google Forms icon by Icons8
Google Sheets icon by Icons8