How YouTube Streams Billions of Videos

Uploading a video is the easy part. What happens in the following minutes is a distributed systems problem most companies never have to solve: taking one source file and turning it into the thing that starts playing instantly, on a 5G phone in Lagos and a dial-up-equivalent connection in a rural area, at the same moment, from a fleet that serves more video than any other site on the internet. YouTube is responsible for a substantial fraction of all downstream bandwidth globally — around a sixth of fixed-network traffic — and nearly every byte of it is delivered through the pipeline behind the play button.

One upload becomes dozens of files

When a creator uploads, the raw bytes land in object storage and a job is dropped onto a durable queue. Stateless transcoding workers pull jobs off that queue and process them in parallel. The critical trick is that a video is not encoded as one unit. It is cut into short chunks aligned to GOP boundaries — a few seconds of video each — and every chunk is encoded independently across a matrix of resolution-and-codec combinations at the same time. A one-hour video that would take six hours to encode serially finishes in roughly ten minutes with enough workers. The pieces are concatenated back together only after the chunks are done.

YouTube does not encode them all eagerly: the lowest rungs are finished first and pushed to the edge so the video is playable within minutes, and the higher rungs — 4K, and beyond — arrive over the following hours. That is why a video you upload is watchable almost immediately, but the 4K version only shows up later.

The economics of that encoding ladder are brutal and interesting. Encoding is expensive: an AV1 encode in software can cost an order of magnitude more than H.264, and at YouTube's scale, the encoding fleet must produce these streams around the clock for hours of uploaded video every minute. Yet the same codec that costs the most to encode is the one that halves delivery bandwidth — and at a network serving a sixth of the world's downstream traffic, halving the bytes pays for the encode many times over. This is why YouTube tiers its ladder by popularity: universal H.264 for everything, AV1 for content watched enough to justify the cost, and high rungs generated on demand for the long tail. A video's quality is not just a technical property; it is an economic decision made per minute of content.

# See the ladder and the manifest a real stream serves
curl -s https://... | grep -o 'RESOLUTION=[0-9]*' | sort -u
# Or probe any DASH MPD you're served
ffprobe -v error -show_format https://.../manifest.mpd | head -20

The manifest, then a sequence of small files

Your player never downloads a video. It downloads a manifest — an MPD for the DASH standard that Chrome, Firefox and Android use, or an m3u8 playlist for the HLS standard that iOS and Safari use — and then a sequence of small segment files. Segments are typically a few seconds long, which is short enough that the player can switch quality frequently and long enough that the download overhead stays negligible. Because each segment is immutable and content-addressed, CDN caching is trivial: the same segment can be served to a million viewers from the same cached copy. The manifest itself is refreshed on a short TTL, so a live stream can keep pointing viewers at newly generated segments.

Adaptive bitrate is a control loop

The decision about which rung to download next is made entirely on the client. The player measures how fast each segment actually arrived and combines that with its buffer occupancy — how many seconds of video it has in reserve — to pick the next quality. A full buffer means it can afford a heavier, better segment; a draining buffer means it must drop to a lower rung before it runs out and stalls. Every few seconds the loop re-evaluates.

• The player measures throughput on every segment it fetches.
• It picks the next segment's quality from a buffer-occupancy heuristic.
• Overreacting causes visible quality flapping; underreacting causes stalls.

Tuning that loop is genuinely hard. Pick too high and a heavy segment depletes the buffer and the video rebuffers; pick too low and the picture is blurry for no reason; oscillate and the quality visibly flaps between rungs. The formal results in this space are elegant: algorithms like BOLA make a provable trade-off between average quality and rebuffering risk using only the buffer as input. In practice, the best players maintain a large buffer target — often around thirty seconds for on-demand — as insurance, so a short bandwidth dip is absorbed without the user ever noticing.

The cache is closer than you think

The reason playback starts instantly is not a fast origin; it is that the origin is essentially inside your ISP. Google supplies caching appliances — the Google Global Cache — at no cost to internet service providers, who host them inside their own networks in exchange for not having to transit the video across the public internet. When your player requests segments, its IP prefix maps to the local cache, and the bytes travel a few kilometres across the last mile rather than across an ocean. For typical ISPs, a large majority of cacheable traffic is served from that on-network cache, and the cache-hit rate on popular on-demand content approaches the high nineties.

The cache hierarchy is deliberate: core data centres feed edge points of presence, which feed the caches inside ISP networks. Content that is popular is served from the innermost layer, and only misses reach further out. That hierarchy, plus the codecs that halve bitrate, is the actual reason a 4K video starts on a phone over cellular without buffering — not luck, and not a magic origin.

The cache hierarchy is deliberate

Peeling back that hierarchy reveals the economics of the whole operation. Core data centres hold the only authoritative copy of each video; edge points of presence sat near peering exchanges; and the innermost tier — Google Global Cache appliances running inside ISP networks — holds the hottest content. When a viewer's IP prefix points to a local cache node, the request is served from inside the ISP's own walls, over a last-mile hop of a few kilometres, at effectively zero transit cost to the carrier. That is why an ISP with no access to a major CDN would pay dearly for every popular video streamed to its subscribers, and why the cash-for-cache arrangement is mutually attractive: YouTube gets near-zero latency, the ISP gets off transit, and the subscriber gets instant playback paid for by no one's frustration.

The system is self-interested at every tier, which is why it scales the way it does. Every tier only stores content because it is efficient to do so; every miss is a deliberate decision that the content was not worth keeping. Reduce the bitrate enough with AV1 and a given cache stores more minutes per byte; make segments immutable and content-addressed and the raw churn of millions of previews becomes cache-agnostic rather than cache-hostile. The whole pipeline — many renditions, short immutable segments, client-driven adaptation, a caching hierarchy anchored in the network the viewer already pays for — is a series of interlocking efficiencies that add up to 'it just plays'.

What the pipeline teaches

The YouTube pipeline is a masterclass in converting an impossible-sounding requirement — start a huge file instantly for a billion concurrent users — into a problem made of small pieces, each independently cacheable, each cheap to move. Transcode once into many shapes and cache everything. Make the client do the quality decisions so the server never has to know the user's bandwidth. Push the bytes as close to the user as physics allows. Every engineering team that streams anything — product video, dashboards, even images at scale — is reinventing some smaller version of this exact architecture.