Talking to clients

Ordinary web traffic is request and response: the client asks, the server answers, and the conversation ends. The server cannot speak first. That is fine for loading a page, but wrong for a chat message, a delivery location or a live score, where the server knows something new and the client does not. Four techniques close that gap, each with a different cost.

In polling the client asks again and again on a timer: "anything new?"

It is simple and works everywhere, but it wastes effort and adds delay. Updates arrive up to one interval late, and most requests return nothing. Do the arithmetic: one million clients polling every 5 seconds is 200,000 requests per second, nearly all of them empty. Shortening the interval improves freshness and multiplies the load, so polling suits slow, low-volume cases.

In long polling the client asks, and the server holds the request open until it has something to send or a timeout expires, often 30 to 60 seconds. The client then processes the answer and immediately asks again.

Updates arrive almost instantly, and idle clients cost far fewer requests, about one per timeout window rather than one per few seconds. The costs are that each waiting client occupies a connection, the server must manage many open requests, and every message ends the exchange so the client must reconnect. Reconnects can bunch up after an outage.

A WebSocket starts as an HTTP request and then upgrades to a persistent, two-way connection. After that, either side can send a message at any time with very little overhead, since there are no repeated headers or handshakes.

It is the natural fit for chat, collaborative editing and games. The price is that connections are stateful: a server remembers who is connected to it, so load balancing, deployment and failure handling are harder than with plain requests. You also need your own heartbeats to detect dead connections, and reconnection logic on the client.

Server sent events (SSE) keep one HTTP response open and let the server push a stream of text messages to the client, one direction only. Browsers reconnect automatically and can resume from the last event id. It is lighter than WebSockets and works over ordinary HTTP infrastructure, which makes it excellent for live feeds, notifications and progress updates. If the client must also send frequent messages, use a WebSocket, or send those with ordinary requests.

• Rare updates and simplicity matter: polling.
• Near-instant updates but you want plain HTTP and minimal change: long polling.
• Frequent two-way messages: WebSocket.
• One-way live stream from server to client: SSE.

Mobile apps in the background cannot hold connections. They rely on the platform's push notification service, which keeps one shared connection and wakes the app.

Open connections cost memory, roughly several kilobytes each, so a million connections need on the order of ten gigabytes plus the work to keep them alive. A single tuned server can hold tens of thousands to a few hundred thousand. Plan for many gateway servers.

That creates a new problem. If Alice is connected to gateway 1 and Bob to gateway 2, how does a message from Bob reach Alice? The standard answer is a publish-subscribe backplane: each gateway subscribes to the users it holds, and the sender publishes to the channel, so whichever gateway has the connection delivers it.

• Heartbeats: send small periodic messages so dead connections are noticed and cleaned up.
• Reconnect with backoff and jitter: after an outage, a million clients reconnecting at once can flatten the servers. Randomize the delay.
• Backpressure: a slow client must not make the server buffer without limit. Drop, batch or disconnect.
• Presence: whether a user is online is derived from heartbeats and is naturally approximate.
• Ordering and delivery: assign message ids so clients can detect gaps and request what they missed. Make handling idempotent, because a message may be delivered twice.

Choose the simplest technique that meets the freshness the product actually needs.