YouTube, Netflix and Typeahead

These two problems sit at opposite ends. Video streaming moves enormous files through a pipeline of expensive processing, and serving them is a bandwidth problem. Typeahead returns five short strings in under 100 milliseconds, and it is a latency problem. The first is solved by processing once, serving from the edge, the second by precomputing the answer.

Users upload videos, and anyone can watch them smoothly on any device and network. Views are far more common than uploads. Watching must start quickly and not stall, uploads must not be lost, and the catalog must scale to billions of videos.

Assume 500,000 new videos a day at about 100 MB each. That is about 50 TB of raw uploads per day. Each video is converted into several versions, so storage grows by a factor of two or three, to around 150 TB a day. The serving side is the real giant: 10 million people watching at once at 3 Mbps is 30 terabits per second. No single data center can send that, which tells us the design must be built on a content delivery network.

A raw upload is useless to most viewers, because its format, size and bitrate may not suit their device or connection. So every upload goes through transcoding.

1. The client uploads the original, often in chunks so a failure resumes cheaply, into a blob store.
2. A message goes onto a queue saying a new original exists.
3. A fleet of transcoding workers splits the video into short segments and converts them in parallel into many renditions: several resolutions (for example 240p to 4K), bitrates and codecs. Parallel work turns hours of processing into minutes.
4. Workers also produce thumbnails, extract audio, and run checks such as copyright and safety detection.
5. The finished renditions are stored, a metadata row is marked ready, and the video becomes visible.

Using a queue lets the system absorb spikes of uploads, and retry a failed worker without restarting everything.

Instead of one large file, each rendition is cut into short segments of a few seconds, with a manifest file listing them. Protocols such as HLS and DASH work this way. The player downloads the manifest, then fetches segments one at a time, measuring how fast each arrives. If the network slows, it switches to a lower quality for the next segment. If the network recovers, it climbs back up. This is adaptive bitrate streaming, and it is why a video degrades gracefully instead of freezing.

A CDN stores copies of popular segments on servers close to viewers. A request goes to the nearest server, which answers from its cache or fetches from an origin and keeps a copy. Because video popularity is extremely skewed, a small part of the catalog accounts for most traffic and caches very well. Services such as Netflix go further and place caching appliances inside internet providers' networks, and pre-load expected popular titles overnight, so evening traffic never crosses the wider internet. Rarely watched videos are served from the origin or a regional tier.

Metadata (title, owner, status, renditions, comments) lives in a database, sharded by video id and cached heavily. View counts arrive at enormous rates, so they are buffered and batched through a stream, not written to a database row per view, and displayed approximately. Recommendations are computed offline from watch history and refreshed periodically, while a fast layer serves the precomputed lists. Search uses a separate index over titles and tags.

As a user types, show the five most popular completions of what they have typed. Suggestions must appear in under about 100 milliseconds, or they feel broken. They should favor popular and recent queries, and slightly stale suggestions are acceptable.

Suppose there are 100 million distinct queries a day worth indexing. At roughly 30 bytes each that is only about 3 GB of text. Even with overhead for the data structure, the whole suggestion index fits in the memory of a few machines. The challenge is speed and load: each character typed can trigger a request, so traffic is many times the search rate.

A trie (prefix tree) stores strings by their characters. Each node stands for a prefix, and children extend it by one letter. Finding the node for a prefix takes time proportional to the length of the prefix, regardless of how many strings are stored.

Finding the node is not enough, because under a short prefix like "a" there may be millions of completions. Searching below the node for the best five would be slow. The standard trick is to store the top five completions at every node, computed ahead of time. Then a lookup is: walk down the prefix, read the stored list, return. That is O(L) for a prefix of length L.

Do not update the trie on every query. Instead, build it offline: log every query, aggregate counts over a period such as an hour or a day, rank them, build a fresh trie with its top-five lists, and swap it in on the serving machines. Real-time freshness for sudden trends can be handled by a small, separate recent-queries layer merged at read time. This keeps the serving path read-only, simple and fast, and moves the heavy work off the critical path.

• Partition the trie by prefix ranges, with the first letters assigned to different machines, but balance by traffic, since letters like "s" are far busier than "x". Replicate every partition for availability and read throughput.
• Cache results for common prefixes at the edge and in the browser.
• Client behavior matters as much as the server: wait for a short pause in typing (debounce about 150 ms), start only after two or three characters, cancel outdated requests, and prefetch likely next results.
• Ranking blends popularity, recency and, if allowed, personal history. Filtering removes offensive or unsafe suggestions before they enter the trie.
• Failure: if a server is down, stale or missing suggestions are acceptable and the search box still works, so typeahead should degrade silently rather than block searching.

• Do expensive work once, ahead of time: transcoding the video, building the trie.
• Serve from cheap, close, cached copies: the CDN edge, an in-memory trie.
• Keep the user-facing path simple, using queues and offline jobs to absorb heavy and bursty work.
• Accept approximate answers where nobody can tell: view counts, suggestion freshness.