The System Design Newsletter

The System Design Newsletter

Share this post

The System Design Newsletter
The System Design Newsletter
How LinkedIn Adopted Protocol Buffers to Reduce Latency by 60%
Copy link
Facebook
Email
Notes
More
User's avatar
Discover more from The System Design Newsletter
Weekly newsletter to help busy engineers become good at system design
Over 152,000 subscribers
Already have an account? Sign in

How LinkedIn Adopted Protocol Buffers to Reduce Latency by 60%

#12: You Need to Read This - Awful JSON Serialization (5 minutes)

Neo Kim's avatar
Neo Kim
Oct 08, 2023
135

Share this post

The System Design Newsletter
The System Design Newsletter
How LinkedIn Adopted Protocol Buffers to Reduce Latency by 60%
Copy link
Facebook
Email
Notes
More
15
9
Share

Get the powerful template to approach system design for FREE on newsletter sign-up:


This post outlines how LinkedIn reduced latency using Protocol Buffers. If you want to learn more, scroll to the bottom and find the references.

  • Share this post & I'll send you some rewards for the referrals.

LinkedIn uses microservices architecture because the number of daily requests they receive is in the range of billions. Also they needed to scale.

But microservices architecture increased their network calls and degraded latency.

They used JavaScript Object Notation (JSON) as the data serialization format but it became a performance bottleneck. So they moved to Protocol Buffers.

This post outlines how LinkedIn adopted Protocol Buffers to reduce latency. I think it’s important to understand data serialization and Protocol Buffers first. So I’ll teach you the basics first.


What Is Data Serialization?

What is data serialization?
A get-together of people who speak different languages

Imagine a get-together of people who speak different languages. They have no choice but to speak in a language that everybody understands.

So each person must translate thoughts from their native to the common language. But it reduces communication efficiency. This is how data serialization works.

Now in computing terms. Translating an in-memory data structure to a format that can be stored or sent across the network. This is called data serialization.

What Is Protocol Buffers?

Protocol buffer (Protobuf) is a data serialization format and a set of tools to exchange data.

Protobuf keeps data and the metadata separate. And serializes data into binary format.

Besides Protobuf messages are sent across network protocols such as REST or RPC. And supports many programming languages: Python, Java, Go, and C++.

What is protocol buffers?
Protobuf data serialization workflow

Here is the Protobuf workflow:

  1. Create a proto file

    And define payload schema: data fields and types.

  2. Compile proto file to language-specific source files

    Compile the proto file using the Protobuf compiler. And create language-specific source files: One file for the client to serialize data. And other for the server to deserialize data.

  3. Create an executable package

    Compile the generated Protobuf source file with the project code.

  4. Serialize or deserialize data

    Serialize data at runtime.

Why Use Protocol Buffers?

JSON serialization became a performance bottleneck at LinkedIn. Because textual format needed extra network bandwidth. And more computing resources to compress data. It resulted in poor latency and throughput.

Also skipping unwanted data fields is not possible while parsing JSON. Because there is no separation between data and metadata.

But metadata in Protobuf allows parsing specific data fields. And makes it a lot more efficient for a big payload.

Their criteria to find a JSON data serialization alternative were:

  • Smaller payload size. Because it reduces bandwidth needs

  • Improved efficiency. Because it reduces latency

  • Support for many programming languages. Because their tech stack was diverse

  • Easy to plug into the existing setup. Because they wanted to reduce the engineering effort

Protobuf satisfied all the criteria. So they moved to Protobuf.

Protobuf reduced their P99 latency by 60% for big payloads. And improved average throughput by 6.25% for response payloads.

99th latency percentile is called P99 latency. Put another way, 99% of requests will be faster than the given latency number. Or only 1% of requests will be slower than P99 latency.

Here is a summary of the Protobuf study by Auth0.com

If services running JavaScript and Java communicate with each other. There is no big performance improvement with Protobuf. And the latency difference was only 4% compared to compressed JSON.

But if services running Java and Python or Java communicated with each other. Protobuf offered 6 times better latency compared to JSON.

In simple words, there is a big performance improvement for Protobuf with non-JavaScript environments.

Protobuf Rollout at LinkedIn

They used Rest.li framework for calls between microservices. Because it abstracted many parts of data exchange: serialization and service discovery.

Protobuf Rest.li framework
Rest.li framework

And this is how they rolled out Protobuf:

  1. Add Protobuf support to Rest.li framework

  2. Increment Rest.li framework version. And redeploy every microservice

  3. Release slowly using client configuration. And reduce service disruption

Protocol Buffers vs JSON

I will outline the top 3 benefits and limitations of Protobuf and JSON because it might help you to make better architectural decisions with your project.

Protobuf benefits:

  • Support for schema validation

  • Improved performance with big payloads. Because it uses the binary format

  • Support for backward compatibility

Protobuf limitations:

  • Hard to debug. And not human-readable

  • Extra effort to update the proto file needed

  • Limited language support compared to JSON

JSON benefits:

  • Easy to use and human-readable

  • Easy to change. Because it provides a flexible schema

  • Support for many programming languages

JSON limitations:

  • No support for schema validation

  • Poor performance for big payloads

  • Backward compatibility problems

Takeaways

Use Protobuf when:

  • Payload is big

  • Communication between non-JavaScript environments needed

  • Frequent changes to the payload schema expected

Use JSON when:

  • Simplicity needed

  • High performance is not needed

  • Communication between JavaScript and Node.js or other environments needed

Protobuf gave big performance improvements for LinkedIn. But it is important to check if Protobuf is best for your use case to prevent over-engineering.


👋 PS - Are you unhappy at your current job?

While preparing for system design interviews to get your dream job can be stressful.

Don't worry, I'm working on content to help you pass the system design interview. I'll make it easier - you spend only a few minutes each week to go from 0 to 1. Yet paid subscription fees will be higher than current pledge fees.

So pledge now to get access at a lower price.

“This newsletter is packed with valuable information, provided in a simple & clear way.” Petar


Consider subscribing to get simplified case studies delivered straight to your inbox:


Author NK; System design case studies
Follow me on LinkedIn | YouTube | Threads | Twitter | Instagram | Bluesky

Thank you for supporting this newsletter. Consider sharing this post with your friends and get rewards. Y’all are the best.

system design newsletter

Share


Thank you for the referral and for helping this community grow.

System Design Case Studies

Get featured in the newsletter: Write your feedback on this post. And tag me on Twitter, LinkedIn, and Substack Notes. Or you can reply to this email with anonymous feedback.


The Polymathic Engineer by

Franco Fernando
is golden. It covers system design. And his writing style is one of the best. Please consider subscribing to his newsletter.

How Giphy Delivers 10 Billion GIFs a Day to 1 Billion Users

How Giphy Delivers 10 Billion GIFs a Day to 1 Billion Users

NK
·
October 12, 2023
Read full story
7 Simple Ways to Fail System Design Interview

7 Simple Ways to Fail System Design Interview

NK
·
October 10, 2023
Read full story

References

  • https://engineering.linkedin.com/blog/2023/linkedin-integrates-protocol-buffers-with-rest-li-for-improved-m

  • https://linkedin.github.io/rest.li/user_guide/server_architecture

  • https://auth0.com/blog/beating-json-performance-with-protobuf/

  • https://programmathically.com/protobuf-vs-json-which-data-serialization-format-is-best-for-your-engineering-project/

  • https://protobuf.dev/


Subscribe to The System Design Newsletter

By Neo Kim · Launched 2 years ago
Weekly newsletter to help busy engineers become good at system design
Sunil Hari's avatar
Anton Zaides's avatar
Raviraj Achar's avatar
Kayode Hadilou ADJE's avatar
Akash Kumar Singh's avatar
135 Likes∙
9 Restacks
135

Share this post

The System Design Newsletter
The System Design Newsletter
How LinkedIn Adopted Protocol Buffers to Reduce Latency by 60%
Copy link
Facebook
Email
Notes
More
15
9
Share

Discussion about this post

User's avatar
Sathish's avatar
Sathish
Oct 14, 2023

Thanks NK for the detailed explanations. Why does JS doesn’t perform as good as Java or python using Protobuf?

Expand full comment
Like (2)
Reply
Share
1 reply by Neo Kim
Raviraj Achar's avatar
Raviraj Achar
Oct 8, 2023

JSON -> Binary, yea totally makes sense for perf wins.

Expand full comment
Like (2)
Reply
Share
13 more comments...
8 Reasons Why WhatsApp Was Able to Support 50 Billion Messages a Day With Only 32 Engineers
#1: Learn More - Awesome WhatsApp Engineering (6 minutes)
Aug 27, 2023 • 
Neo Kim
741

Share this post

The System Design Newsletter
The System Design Newsletter
8 Reasons Why WhatsApp Was Able to Support 50 Billion Messages a Day With Only 32 Engineers
Copy link
Facebook
Email
Notes
More
24
How PayPal Was Able to Support a Billion Transactions per Day With Only 8 Virtual Machines
#30: Learn More - Awesome PayPal Engineering (4 minutes)
Dec 26, 2023 • 
Neo Kim
248

Share this post

The System Design Newsletter
The System Design Newsletter
How PayPal Was Able to Support a Billion Transactions per Day With Only 8 Virtual Machines
Copy link
Facebook
Email
Notes
More
14
How Stripe Prevents Double Payment Using Idempotent API
#45: A Simple Introduction to Idempotent API (4 minutes)
May 9, 2024 • 
Neo Kim
382

Share this post

The System Design Newsletter
The System Design Newsletter
How Stripe Prevents Double Payment Using Idempotent API
Copy link
Facebook
Email
Notes
More
30

Ready for more?

© 2025 Neo Kim
Publisher Privacy
Substack
Privacy ∙ Terms ∙ Collection notice
Start writingGet the app
Substack is the home for great culture

Share

Copy link
Facebook
Email
Notes
More

Create your profile

User's avatar

Only paid subscribers can comment on this post

Already a paid subscriber? Sign in

Check your email

For your security, we need to re-authenticate you.

Click the link we sent to , or click here to sign in.