Rate limiting

The shape

A rate limiter bounds work over time. It protects shared capacity from accidental bursts, noisy neighbors, and clients that retry too aggressively.

The useful question

Where does the limit live? A local counter is fast but differs across replicas. A shared counter is consistent but adds a dependency to the request path.

Stress test

Raise one client's request rate beyond its token budget. Watch the service return a deliberate rejection rather than letting the queue grow without bound.

Tradeoff

Token bucket permits a short burst while keeping a long term rate. Choose a burst size that reflects the real service capacity, not only an arbitrary number.