Share this letter & I’ll send you some rewards for the referrals.
50 million people hit play, and your live streaming system has only a few seconds to respond…!
A live streaming platform must capture a high-quality video feed, process it in near real time, and deliver it to millions of viewers across phones, laptops & TVs.
Traffic could surge within seconds when a match starts/key moment occurs.
Every stage adds latency, and every bottleneck puts the stream at risk.
In this newsletter, we’ll break down the system design of a live streaming platform built for 50 million concurrent viewers.
Let’s start with the system requirements…
A walkthrough of every agent PR
Every agent PR comes with a screenshot. Agents open them faster than anyone can read them.
PR Lens adds the architecture. It draws each pull request as a canvas of your system, marks every part new, changed, or removed, and walks you through how it fits.
On every PR, you see:
What changed, marked on the parts of the system it touches
What else it reaches, including services the diff never names
How a request moves through the system now
What could break, flagged before you merge
2,200+ GitHub stars and 45,000+ pull requests drawn.
(Thanks to Coldtea for partnering on this newsletter.)
I want to introduce Mandeep Singh as a guest author.
He’s an author, educator, and software engineer with a strong passion for building simple, scalable systems.
He authored the O’Reilly book “System Design on AWS” and is working on the upcoming O’Reilly book “System Design From Scratch”. He also supports others’ learning and development by sharing lessons on Cloud and System Design on YouTube.
Checkout his work:
System Design on AWS Book (Also find on Amazon)
System Requirements & Scale
Before you design the architecture, you must define what the system supports.
i.e., separate functional requirements from non-functional requirements
Functional requirements describe what viewers can do, while non-functional requirements define how well the system must perform at scale.
Let’s get into it:
Functional and Non-Functional Requirements
Here are the key “functional” requirements:
Viewers can watch live matches.
Video must be streamed based on device type & network bandwidth.
Let’s leave out these features to limit the scope:
Multiple audio languages (keep only English),
Live chat & user reactions,
Ads pipeline.
“Non-functional” requirements define how well the platform must perform. Here’s an essential list for smooth operations:
Low latency: the stream should reach viewers in real time without buffering. For this design, we’ll target 4 to 30 seconds of end-to-end latency.
Scalability: traffic could exponentially rise/fall sharply during a live event; the system must absorb sudden spikes in concurrent viewers.
Extensibility: architecture should support new event types & device types without major redesigns.
End-to-end latency comes from several stages between capture & playback.
(We’ll examine those stages separately later & see where each one adds delay.)
Latency is only half the problem… Scale also drives many architecture choices…
So let’s put some numbers behind it…
System Scale
Live events create sharp traffic spikes…
i.e., peak concurrent viewers matter more than average traffic.
For this design, assume:
Maximum concurrent viewers: 50 million
Average stream bitrate: 5 Mbps
(Bitrate is the amount of video data transferred each second.)
So the total outgoing bandwidth for a 1080p video is:
Total Bandwidth = Peak Viewers × Average Bitrate
50,000,000 × 5 Mbps = 250,000,000 Mbps = 250 TbpsThis massive number shows why a central server cannot efficiently deliver the stream to every viewer1.
You need video compression, adaptive bitrate streaming, and a Content Delivery Network (CDN) to distribute the traffic across edge locations closer to viewers.
Let’s move on to the high-level architecture…
High-Level System Design
The system streams video from cameras at the event venue to users’ screens.
There are three main phases:
Video creation/ingestion,
Video processing,
Video distribution.
Here’s the high-level workflow:
High-resolution cameras record the video at the event venue.
The system ingests the video at the venue and prepares it for transfer to application servers for extra processing.
Video processing prepares the stream to support different device types.
Finally, video distribution delivers the stream to users’ devices.
Each phase addresses a different part of the journey from camera to screen.
Let’s start with video ingestion…
Video Ingestion
The journey starts with cameras at the live event…
These cameras capture high-quality video and create a large amount of data. Transferring the original video directly for further processing would require significant bandwidth & take more time. So the system first reduces the video size while keeping the quality in check.
A video “encoder” at the event location handles this step.
The event location is typically called the Production Control Room (PCR), although the exact terminology varies by company.
The encoder uses video codecs to compress the video.
Here are the popular options:
H.264 - Advanced Video Coding (AVC).
H.265 - High Efficiency Video Coding (HEVC), the successor to H.264.
Once encoded, the smaller video stream needs to reach a central location for further processing & delivery. This infrastructure could run in the cloud or in your own on-premises data center.
Next comes an important design decision: how do you transfer this live video stream with low latency?
Live Data Transfer
Once the video gets encoded, the system must transfer it from the PCR to a central processing location.
Think of it like uploading a large video file to Google Drive:
The upload takes time, especially for large files. But for live streaming, video must reach the processing servers with minimal delay.
Transferring video over the public internet creates 2 challenges:
Variable network conditions: Network congestion & packet loss could delay video delivery.
Bandwidth limitations: Available bandwidth could fluctuate, especially during peak traffic.
For major live events, a “dedicated” network connection helps address these challenges.
A dedicated connection links the event location to the central processing infrastructure without relying on the public internet. It provides more predictable bandwidth & network performance.
Yet it cannot eliminate every source of delay/failure. Plus, setting up and managing dedicated network links requires significant effort.
Cloud providers offer services to simplify this process:
AWS Direct Connect: Provides dedicated network connectivity between your infrastructure and AWS.
Azure ExpressRoute: Provides private connectivity between your infrastructure and Microsoft Azure.
Both services help establish more predictable network paths for transferring live video to cloud infrastructure.
But network connectivity is only part of the solution. You also need a protocol to transport a continuous video stream efficiently.
Let’s look at the networking protocols used for live video transfer…
Networking Protocols
Live video requires a continuous data flow between the event location & central processing servers.
In a typical HTTP request, the client sends a request, receives a response, and completes the exchange. But repeatedly establishing connections for a continuous video stream would introduce unnecessary overhead.
A “persistent” connection helps avoid this problem.
WebSockets might seem like an obvious choice… They support persistent, bidirectional communication between clients and servers.
But WebSockets do not provide built-in media transport features such as video codecs, timestamps, or packet-loss recovery. You’d need to implement additional mechanisms to handle live audio & video reliably.
WebSockets are better suited to features such as live chat & viewer reactions.
For transporting live video, we need protocols designed specifically for media delivery.
Real-Time Messaging Protocol
Real-Time Messaging Protocol (RTMP) is a protocol designed to transfer audio & video streams with low latency.
(Historically, RTMP supported video delivery from streaming servers to client devices through Adobe Flash Player. But Flash Player reached end of life in December 2020.)
RTMP remains widely used for the first phase of live streaming: transferring video from the event location to ingestion servers.
Here’s how it works:
Client initiates a handshake with the RTMP server.
After the handshake succeeds, the client establishes a persistent connection.
RTMP divides media stream into smaller chunks & transfers them over connection.
The server receives the chunks & reconstructs the media stream for further processing.
RTMP runs over TCP, which provides reliable, ordered data delivery.
Yet TCP introduces a trade-off:
When packets get lost, TCP retransmits them. Subsequent data may be delayed until the missing data arrives, a behavior known as head-of-line blocking. And this could increase latency or cause playback interruptions under poor network conditions.
A dedicated network connection helps reduce congestion and packet loss, thus making RTMP a practical choice for transporting live video from the event venue.
Another limitation is codec compatibility:
Traditional RTMP implementations primarily support H.264 video, while H.265 support requires newer extensions and compatible software.
Secure Reliable Transport
Secure Reliable Transport (SRT) is a protocol designed for low-latency video transport.
SRT runs over UDP. SRT adds reliability mechanisms on top of UDP, including packet-loss detection, selective retransmission, and configurable latency buffers.
Here’s the key difference:
RTMP: Uses TCP to deliver data reliably and in order. Packet loss can delay subsequent data while TCP recovers missing packets.
SRT: Uses UDP with selective retransmission. It attempts to recover lost packets within a configured latency window and avoids waiting indefinitely for packets that arrive too late.
This makes SRT useful when transferring live video over networks with fluctuating bandwidth or packet loss.
Also RTMP has broad support for existing streaming hardware & software. But SRT support depends on the capabilities of the encoder, decoder, and streaming infrastructure.
Both protocols primarily transport video from the source to the streaming backend.
But delivering video from the backend to millions of viewers requires a different approach…
HTTP Live Streaming
HTTP Live Streaming (HLS) is a streaming protocol by Apple.
Instead of maintaining a continuous media connection between the streaming server and every viewer, HLS divides encoded video into smaller media segments.
Here’s what a typical HLS workflow looks like:
The encoder continuously produces compressed video.
A packager divides the encoded stream into media segments, typically 2 to 4 seconds long.
The system creates a playlist, called a manifest, containing references to available segments.
The video player downloads the playlist and requests segments over HTTP.
As new segments become available, the player continues downloading them to maintain playback.
The camera does not need to stop recording after each segment. Video capture and encoding continue while the packager creates segments from the incoming stream.
Yet segment duration introduces an important trade-off.
Shorter segments could reduce live-stream latency but increase request overhead & place extra demands on the streaming infrastructure. Longer segments reduce request frequency but increase playback delay.
Low-Latency HLS extends this approach by letting players access partial segments before a segment is complete.
(Netflix uses HLS with 2-second segments for its live streaming architecture.)
HLS also offers broad compatibility with modern playback devices and CDNs.
For our architecture, we can use RTMP/SRT to transport video from the event location to the central processing servers, then use HLS to distribute the processed stream to viewers.
If ultra-low-latency playback becomes a requirement, we should evaluate alternatives such as WebRTC or specialized low-latency delivery protocols. SRT remains a strong option for contribution feeds, but it is not a direct replacement for HLS across millions of consumer devices.
Video Delivery Workflow
Let’s summarize the first phase of our architecture:
Video capture: Multiple high-resolution cameras record the event from different angles.
Production Control Room: Camera feeds reach the PCR or operations center, where the production system inspects and selects the video streams for further processing.
Encoding and compression: Once the production feed is finalized, video encoders compress it using codecs such as Advanced Video Coding (AVC/H.264) or High Efficiency Video Coding (HEVC/H.265). This reduces the bandwidth required for transmission. Encoding can happen on-site with hardware or software encoders, or through cloud-based services. For example, AWS Elemental MediaLive provides live video encoding & processing in the cloud.
Video transfer: The compressed stream travels from the event location to centralized processing infrastructure over a dedicated network connection, using a protocol such as RTMP/SRT.
Choosing Video Segment Size
One final consideration is how the continuous video stream gets divided into segments for downstream processing & delivery.
The segment duration affects several factors:
Live-stream latency: Shorter segments reduce the delay between the live event and viewer playback.
Encoding and packaging: Segment boundaries must align with the encoded video structure, and the system needs time to prepare segments for delivery.
Network overhead: Smaller segments generate more frequent HTTP requests, increasing request volume across clients, servers, and CDNs.
For conventional HTTP-based live streaming, segment durations range from around 2 to 10 seconds, depending on the configuration.
The appropriate value depends on the event’s latency requirements, encoding configuration, network conditions, and geographic distribution of viewers. Once the video gets transferred to the central servers, the next phase prepares it for playback across different devices and network speeds.
Let’s examine video processing next…
Video Processing
Once the video stream reaches the central server, next step is to prepare it for different devices & network conditions.
A viewer watching on a mobile phone with a slow internet connection has different requirements from someone watching on a 4K television with high-speed broadband.
The system must support both without interrupting playback.
Transcoding converts the incoming video stream into multiple encoded versions, each with a specific resolution, bitrate, and codec.
Two important parameters determine the output:
Bitrate: The amount of video data transmitted per second, typically measured in megabits per second (Mbps). Higher bitrates allow better video quality but require more bandwidth.
Resolution: The number of pixels in each video frame, such as 4K, 1080p, 720p, or 480p. Higher resolutions contain more visual detail but typically require higher bitrates to maintain quality.
For example, an MP4 video could be transcoded into many quality levels:
High bitrate: 4K, 1080p, or 720p.
Medium bitrate: 1080p, 720p, or 480p.
Low bitrate: 480p, 360p, or 144p.
Resolution & bitrate are separate parameters, so each output needs its own bitrate configuration. These versions form an adaptive bitrate (ABR) ladder.
The video player then selects the appropriate version based on the viewer’s available bandwidth, device capabilities, and playback conditions.
i.e., someone watching on a fast Wi-Fi connection might receive a 1080p stream. If their connection slows down, player can switch to 720p/480p to reduce buffering.
Managed Services vs Custom Transcoding
At this stage, you must decide whether to build your own transcoding infrastructure or use a managed service.
Cloud providers offer services for video encoding & format conversion.
AWS Elemental MediaConvert converts video files into different formats, resolutions, and bitrates. It’s designed for file-based video processing.
For continuous live streams, AWS Elemental MediaLive is an appropriate managed service because it supports real-time video encoding & transcoding.
The output is a collection of video renditions ready for packaging & distribution.
Video Packaging: HLS and DASH
After transcoding, the system must package the video into a format supported by viewers’ devices.
Here are 2 popular protocols:
HTTP Live Streaming (HLS): Widely supported across modern streaming platforms.
Dynamic Adaptive Streaming over HTTP (DASH): An open standard for delivering adaptive video streams over HTTP.
Both support adaptive bitrate streaming, thus allowing video players to switch between quality levels as network conditions change.
HLS is commonly used on Apple devices, while many other players and platforms support DASH. Also HLS works on many non-Apple devices, so the choice depends on player and platform compatibility.
Let’s examine how HLS works…
How HLS Organizes Video Segments
HLS divides the encoded video into smaller media segments and generates playlists that tell the player where to find them.
Each segment is typically stored in one of two formats:
.ts: MPEG-2 Transport Stream, a traditional HLS segment format..m4s: Fragmented MP4, commonly used with modern HLS implementations.
The system also generates two types of playlist files, both using the .m3u8 extension.
1. Master Playlist
The master playlist lists the available video renditions, such as 4K, 1080p & 720p.
Each entry includes required bandwidth, resolution, and codec.
The player uses this information to identify compatible renditions and select an appropriate quality level.
2. Media Playlist
Each rendition has its own media playlist.
This playlist contains an ordered list of video segments, along with their durations and locations.
For a live event, the playlist updates as new segments become available. The player repeatedly fetches the latest playlist & downloads appropriate segments.
This structure allows the player to switch between quality levels during playback without restarting the stream.
Video Storage
The system also needs to store video for later playback.
We can retain both the original ingested video & transcoded outputs.
This serves 2 purposes:
Original video: Preserves source material for future processing or re-encoding.
Transcoded video: Allows viewers to replay the event without requiring the system to transcode it again.
Object storage is a common choice for storing these large video files.
Amazon Simple Storage Service (S3) stores the original recordings, transcoded renditions, media segments, and manifests.
We also maintain a metadata store to map each event and video rendition to its corresponding file locations in Amazon S3.
When someone wants to watch a completed event, the system uses this metadata to locate the appropriate video assets.
Digital Rights Management
Live sports content typically comes with strict licensing & distribution requirements.
The platform must protect video streams from unauthorized access & playback.
Digital Rights Management (DRM) encrypts video content and requires an authorized player to obtain a valid license before decrypting and playing it.
Here are the popular DRM systems:
Google Widevine: Supported across Android & many browser-based playback environments.
Microsoft PlayReady: Used across various streaming devices, browsers & television platforms.
Apple FairPlay Streaming: Used to protect HLS content on supported Apple devices.
AWS Elemental MediaPackage can package live video into HLS/DASH formats and integrate with DRM systems to encrypt content.
But DRM license issuance and key management require integration with a compatible DRM provider or license service.
This ensures authorized viewers access protected streams through supported players.
From Video Processing to Distribution
Let’s summarize what happens after the live video reaches the central processing servers.
Transcoding: The incoming stream gets converted into multiple resolutions and bitrates to support different devices & network conditions.
Packaging: Each rendition gets divided into media segments and organized into HLS/DASH manifests.
Storage: Original recordings, transcoded video, and metadata get stored for live delivery & future playback.
Content protection: DRM encryption and license services help restrict playback to authorized viewers.
The processed video is now ready for distribution.
But our system must deliver it to 50 million concurrent viewers without overwhelming the central servers. This is where a Content Delivery Network (CDN) becomes essential.
Let’s examine video distribution next…
Video Distribution
The video is now encoded, transcoded & packaged.
The next challenge is delivering it to 50 million concurrent viewers.
Sending every video segment directly from the central servers would require enormous bandwidth & put massive pressure on the infrastructure.
A CDN caches video segments at edge locations closer to viewers. When thousands of people in the same region watch a match, they receive the same video segments from a nearby edge server instead of repeatedly requesting them from the origin.
But caching introduces a tradeoff:
Cached content could become stale, while live streaming requires viewers to receive new segments with minimal delay. The system must balance cache efficiency with live-stream freshness.
Live video segments are typically immutable once published. So this makes them suitable for caching, while frequently updated manifests require shorter cache durations.
How CDN Distribution Works
Once the video gets transcoded & packaged, the distribution system makes video segments and manifests available through the CDN.
In a typical CDN architecture, edge servers retrieve content from the origin if it’s not already cached. Subsequent requests for the same segment get then served directly from the edge.
Here’s how the process works:
Video packaging: The processing system generates video segments and HLS/DASH manifests.
Origin storage: Packaged content becomes available on an origin server or object storage service.
CDN retrieval: When an edge location receives a request for uncached content, it fetches the required segment from the origin or upstream CDN cache.
Edge caching: CDN stores the segment temporarily so nearby viewers can access it without another origin request.
Video delivery: Viewers download the segments from edge locations through Internet Service Providers (ISPs) or mobile networks.
For live sports, viewer demand concentrates in specific geographic regions.
The distribution system must account for these traffic patterns & ensure sufficient CDN capacity in regions where large audiences are expected.
Here are CDN solutions that support this architecture:
Amazon CloudFront: delivers content through a globally distributed network of edge locations.
Cloudflare CDN: A distributed caching & content delivery service.
Netflix Open Connect: Netflix’s own CDN infrastructure, designed to deliver video efficiently to its viewers.
The goal is to serve video from locations close to viewers while reducing traffic to the central processing infrastructure.
Video Pipeline Coordination
Consider 3 major phases we’ve covered:
Video ingestion
Video processing
Video distribution
These phases depend on one another…
The system must receive the incoming video before it can process it. Transcoding begins as encoded video becomes available, and distribution starts as soon as the required segments and manifests are ready.
For live streaming, these stages operate continuously as a pipeline rather than waiting for the entire event recording to finish.
But failures can occur at any stage…
The system might generate the 720p rendition while the 1080p transcoding process fails.
An orchestration layer helps coordinate the services involved in the pipeline. Its responsibilities include:
Task coordination: Ensure ingestion, transcoding, packaging, and distribution happen in the correct order.
Failure handling: Detect failed processing tasks & trigger necessary recovery actions.
Dependency management: Track which outputs are ready for downstream services.
Distribution readiness: Ensure video segments & manifests become available to CDN as processing progresses.
If the 1080p rendition fails, the system can attempt recovery while continuing to deliver other available renditions.
This prevents a failure in one quality level from unnecessarily interrupting the entire live stream.
Bringing Video Closer to Viewers
The network path between origin infrastructure & CDN locations affects delivery performance.
CDN providers use private backbone networks, peering connections, and optimized routing to transfer content efficiently.
Yet the exact network path depends on the CDN provider and deployment architecture. Traffic does not always travel only over dedicated private connections.
Once the content reaches an edge server, viewers access it through their ISPs or mobile networks.
This final delivery path uses the public internet.
To reduce latency and network congestion, CDN providers place caching infrastructure close to major concentrations of viewers.
Netflix takes this approach with Open Connect.
Its Open Connect Appliances are caching servers that can be deployed inside participating ISP networks. This lets Netflix serve content closer to subscribers while reducing traffic across upstream networks.
AWS offers a related capability through Amazon CloudFront Embedded Points of Presence (POPs).
These smaller CDN locations extend content delivery infrastructure deeper into ISP networks, bringing cached content closer to end users.
Both approaches help reduce the distance video data must travel before reaching viewers.
For a live event with 50 million concurrent viewers, this distributed architecture is essential for handling traffic without overwhelming the origin servers.
The CDN delivers the video segments…The next step is understanding how a viewer’s device discovers the stream, selects the appropriate quality & plays video.
Let’s examine video playback next…
Video Playback
The player must locate the live stream, select the right video quality & download the segments needed for playback.
Here’s the end-to-end request workflow:
Request the stream: App/site sends a request to the playback service, along with the event details.
Retrieve the master manifest: Playback service provides the master manifest URL, which the player uses to retrieve the available video quality options.
Select video quality: The player chooses an initial quality level based on the available network bandwidth & device capabilities.
Retrieve the media manifest: The player requests the media manifest for the selected quality. This manifest contains references to video segments available through CDN.
Download & play: The player downloads the video segments from the CDN, buffers enough data to begin playback, and starts displaying the live stream.
NOTE: We’ve intentionally excluded services responsible for user authentication and authorization, which control access to specific content. Our focus is on delivering live video to viewers with minimal delay.
How Adaptive Bitrate Streaming Works
The viewer’s network conditions could change throughout the event… Someone watching on a mobile device might move from a fast Wi-Fi connection to a slower cellular network.
If the player keeps downloading high-bitrate video segments, playback could stall because the network cannot deliver data fast enough.
The video player continuously monitors factors such as available network throughput and playback buffer levels with adaptive bitrate (ABR) streaming. It then uses this information to select an appropriate video rendition.
When bandwidth increases: Player switches to a higher-bitrate rendition for better video quality.
When bandwidth decreases: The player switches to a lower-bitrate rendition to reduce buffering.
When network conditions stabilize: Player continues downloading segments at a suitable quality level.
These switches typically happen at segment boundaries, allowing the player to change video quality without restarting playback.
The goal is to maintain smooth playback while delivering the highest practical video quality for each viewer.
We’ve now covered the complete journey from cameras at the event venue to video playback on viewers’ devices.
But supporting 50 million concurrent viewers creates extra challenges, especially when traffic increases sharply during a major live event.
Let’s examine how the architecture handles these situations…
Reminder: this is a teaser of the subscriber-only newsletter, exclusive to my golden members.
When you upgrade, you’ll get:
High-level architecture of real-world systems.
Deep dive into how popular real-world systems work.
How real-world systems handle scale, reliability & performance.
(If this newsletter has helped you become a better software engineer, consider subscribing to support my work.)