Messenger and Twitter

Both products move small pieces of text between people, but under opposite pressures. A messenger delivers a message to one person or a small group, quickly and exactly once in order. A Twitter-like service publishes a post once and makes it visible to millions, who read it later in a timeline. Messaging is about delivery, and timelines are about fan-out.

One-to-one and group chat, delivery status (sent, delivered, read), online presence, message history that syncs across devices, and sharing of photos. Messages must never be lost, must appear in order within a conversation, and should arrive in well under a second when the recipient is online. Offline users get their messages when they return.

Assume 500 million daily users sending 40 messages each. That is 20 billion messages per day, or about 230,000 per second. At about 100 bytes of text plus metadata, that is roughly 2 TB per day. Messaging is write heavy and constant, unlike most read-heavy designs, so storage and write throughput lead the design.

Polling is hopeless at this scale. Each client keeps a WebSocket open to a gateway server, so the server can push a message the instant it arrives. A gateway holds a few hundred thousand connections at most, so we run many and keep a mapping from user to the gateway they are connected to, in a fast shared store.

Alice sends a message to Bob.

1. Alice's client sends it over its socket to a gateway, with a client-generated message id.
2. The chat service persists the message first, assigning it the next sequence number in that conversation.
3. It acknowledges to Alice, who shows one tick (sent).
4. It looks up Bob's gateway. If he is online, it pushes the message to him. His client acknowledges, and Alice sees two ticks (delivered).
5. If Bob is offline, the message stays stored, and a push notification is sent through the phone platform. When Bob reconnects, his client asks for everything after the last sequence number it has.

Persisting before delivering is the safety rule: a crash after step 2 loses nothing, since delivery can be retried from storage.

Networks retry, so a message may be sent twice. Delivery is at least once, and the system removes duplicates using the client-generated message id, making processing idempotent. Order is guaranteed within a conversation, not globally: each conversation has its own increasing sequence number, assigned by whoever owns that conversation. That is cheap, since there is no global counter, and it is exactly what people care about.

Access is always "messages of conversation X, newest first, a page at a time." A wide column store fits: partition by conversation id and sort within the partition by message sequence. A conversation lives on one shard, reads are a contiguous range scan, and the store scales by adding machines. Recent messages can be cached. Media goes to object storage with only a reference in the message.

A group message is stored once and delivered to each member, so fan-out equals group size, which is small and bounded. Keep group size limited, because fan-out cost is multiplied by membership. Presence comes from heartbeats: each connected client pings periodically, and a user is shown online while their last heartbeat is recent. Presence is approximate by design, and broadcasting it to everyone is wasteful, so send it only to friends who are currently looking at the conversation. End-to-end encryption means servers store ciphertext only, which limits server-side features such as search.

Users post short text, follow others, and see a home timeline of recent posts from those they follow. They can retweet, like, and search. Assume 300 million daily users and 100 million new tweets per day.

• Writes: 100M / 86,400 is about 1,200 tweets per second.
• Reads: if each user refreshes their timeline about 5 times a day, that is 1.5 billion reads, about 17,000 per second.
• Storage: roughly 1 KB per tweet including metadata is 100 GB per day, about 36 TB per year. Text is small. Media is stored separately in object storage.

Three core pieces. Tweets: id, author, text, time, counts. The social graph: who follows whom, stored as two lookups, followers of X and accounts X follows. Timelines: for each user, a list of tweet ids to show. Tweet ids must be unique and sortable by time, so use the timestamp plus shard plus sequence scheme from the Instagram lesson.

This is the same fan-out tradeoff as before.

• Pull: on each read, fetch recent tweets from everyone you follow and merge. Simple writes, expensive reads, slow for people who follow many accounts.
• Push: on each tweet, append its id to the precomputed timeline of every follower. Reads become one cache lookup. But a user with 50 million followers causes 50 million writes per tweet.
• Hybrid: push for ordinary accounts, and for very popular accounts skip the push and merge their recent tweets in at read time.

Timelines are kept as short lists of tweet ids, for example the latest several hundred, in an in-memory store. Fan-out is done by background workers reading from a queue, so posting returns immediately and a large fan-out never blocks the author.

Likes and retweet counts change constantly on hot tweets. Updating one row from many servers makes it a hot spot, so counters are sharded: split one logical counter into several sub-counters and sum them on read, or buffer increments and write in batches. Exact precision matters little, so eventual accuracy is fine.

Trending topics are a streaming problem: count hashtags over a sliding window and rank by growth rather than raw size, using approximate counting structures to save memory. Search needs an inverted index from words to tweets, updated as tweets arrive, which is its own system.

• Messenger is write heavy, needs ordering and acknowledgements, and uses persistent connections and a wide column store keyed by conversation.
• Twitter is read heavy, uses fan-out and timeline caches, and has a skewed graph where a few accounts dominate.
• Both separate media from text, shard by a key that matches the access pattern, and use queues to keep the user-facing path fast.