All blueprints

BLUEPRINT / TRAFFIC AND SCALE

Load balancing

Spread requests across healthy replicas and decide what happens when one stops answering.

The shape

A load balancer accepts traffic and chooses a healthy service instance. The replicas can remain stateless, which makes replacement and horizontal scaling straightforward.

The useful question

How does the balancer know an instance is useful? A process can answer a health check while its database pool is exhausted. Check the dependency and the user visible path, but avoid making every health probe an expensive request.

Stress test

Increase traffic, then slow one replica. Compare round robin with least connections. A slow instance can hold work longer, so equal request counts do not always mean equal load.

Tradeoff

More replicas add capacity only when the constrained dependency can serve them. Measure the full request path before adding instances.

LIVE SYSTEM / REV 01

Read path under load

Operational
Live service architectureA client sends requests through a load balancer to two app servers, a cache, a database, and a queue. Increase traffic or select a node to inject a failure.HEALTHYClientHEALTHYLoad balancerHEALTHYApp server AHEALTHYApp server BHEALTHYCacheHEALTHYDatabaseHEALTHYQueueEDGE CLIENTSCOMPUTE POOLPERSISTENCE

36k requests per second are balanced across two app servers. The cache absorbs 82% of reads.

Playground · available soon