Tech Stack Evolution at Levels.fyi
#8: Read Now - No Backend to Serving 2.5 Million Unique Users a Month (4 Minutes)
Get the powerful template to approach system design for FREE on newsletter sign-up:
Levels.fyi lets you compare career levels and compensation packages across different companies. It started as a side project and now serves 2.5 million unique users a month.
Share this post & I'll send you some rewards for the referrals.
This post outlines the startup lessons from Levels.fyi. If you want to learn more, scroll to the bottom and find the references.
Dushyant from Levels.fyi reviewed this post before publishing - thanks.
Levels.fyi Tech Stack Evolution
Their tech stack went from no backend to a proper backend. Here is an overview of their tech stack evolution:
Levels.fyi V0 🟩
They used Google Forms and Google Sheets.
Google Forms collected data. While Google Sheets stored data.
They relied on Sheets API to query data. And backed up Google Sheets for durability.
They used Google Forms and Google Sheets because they wanted:
Avoid having to build their own user interface only to collect data.
Avoid extra effort in processing and maintaining data.
Avoid extra tools to analyze, clean, and validate data.
Avoid the operational and maintenance effort.
Avoid the need for data schema management, data migration, and ETL.
Familiarity with Google Sheets.
Easy to use.
Save money.
In-built support for data access control.
Levels.fyi V1 🟩🟨
They built a web UI to serve read requests.
Besides they used lambda functions to read data from Sheets and store it in S3 as JSON files.
They didn’t use a front-end framework. Instead plain JavaScript, jQuery, CSS, and HTML.
Yet their architecture had these drawbacks:
Lack of support for SQL-based data analytics. And it limited their ability to make data-driven decisions.
Sheets API rate limited their queries.
Lambda functions timed out on processing large amounts of data.
Levels.fyi V2 🟩🟨🟧
They wrote the core backend services in Node.js (Nestjs). And used a static site generator.
They didn’t process graphs or statistics shown on the server side. Instead offloaded it to the browser. And it allowed them to scale with minimal resources.
Also they built the API server using API Gateway and EC2. It helped to reduce latency and improve performance compared to V1.
And lambda functions aggregated data and analytics.
They used the backend service for:
Spam detection.
Rate limiting.
Preventing duplicate values.
Levels.fyi V3 🟩🟨🟧🟥
They moved the salary data out of Google Sheets. And built the infrastructure on the AWS stack.
AWS RDS (Postgres) stored critical data. While Metabase generated charts and visuals for internal use by analysts. The lambda functions aggregated the data (for example, percentile) by querying RDS.
They run Node.js (TypeScript) on the server. And orchestrate 5 microservices using a service mesh. The database gets replicated across many regions.
And use TinyStacks for continuous deployments.
But Google Sheets is still used in production for other use cases. There is a 10 million cell limit on Sheets. So they sharded the Sheets as a workaround. Creating many folders and workspaces allowed sharding.
Takeaways
Here are the important lessons from this case study:
1. Keep It Simple
They wanted to focus on what's important - the product idea. So, they used no-code tools and avoided buzzwords. This allowed faster iteration.
2. Avoid Premature Optimization
They wanted to test their product idea. And not invest lots of time and money in doing it.
They used the AWS free tier with credits for the first 2 years. And moved to the paid tier after taking off.
3. Nothing Is Sacred
They outsourced their development and operational burden to Google and AWS. And changed the tech stack to meet the needs.
👋 PS - Are you unhappy at your current job?
While preparing for system design interviews to get your dream job can be stressful.
Don't worry, I'm working on content to help you pass the system design interview. I'll make it easier - you spend only a few minutes each week to go from 0 to 1. Yet paid subscription fees will be higher than current pledge fees.
So pledge now to get access at a lower price.
“An excellent newsletter to learn system design through practical case studies.” Franco
Consider subscribing to get simplified case studies delivered straight to your inbox:
Thank you for supporting this newsletter. Consider sharing this post with your friends and get rewards. Y’all are the best.
References
Dushyant Sabharwal (February 2023). How Levels.fyi scaled to millions of users with Google Sheets as a backend. [online] www.levels.fyi. Available at: https://www.levels.fyi/blog/scaling-to-millions-with-google-sheets.html [Accessed 19 Sep. 2023].
www.linkedin.com. (n.d.). Dushyant S. on LinkedIn: #engineering #startups | 16 comments. [online] Available at: https://www.linkedin.com/posts/dushyantsabharwal_engineering-startups-activity-6978282092782616576-VAIT/ [Accessed 19 Sep. 2023].
www.linkedin.com. (n.d.). We hit 10M cell limit on Google Sheets. Thanks Tanishq Singh! | Zuhayeer Musa posted on the topic | LinkedIn. [online] Available at: https://www.linkedin.com/posts/zuhayeer_googlesheets-salaries-salarytransparency-activity-7102715304924905472-fFH0 [Accessed 19 Sep. 2023].
www.youtube.com. (n.d.). Building a new website using Google Sheets and Google Forms. [online] Available at YouTube [Accessed 19 Sep. 2023].
www.youtube.com. (n.d.). How Levels.fyi Scaled to Millions of Users with Google Sheets. [online] Available at YouTube [Accessed 19 Sep. 2023].
Google Forms icon by Icons8
Google Sheets icon by Icons8