Scalability, reliability and availability

A system is scalable if it keeps working well as load grows: more users, more data, more requests. There are two ways to add capacity.

Vertical scaling means a bigger machine: more CPU, more memory, faster disks. It is simple, because nothing in the software changes. But it has a ceiling, the price rises faster than the power, and one big machine is still a single point of failure.

Horizontal scaling means more machines doing the same job. It has almost no ceiling and it survives the loss of a machine. The price is complexity: the machines must share work and agree on data. For this to work, servers should be stateless, meaning any server can answer any request because nothing important lives only in its memory.

A fault is one part misbehaving: a disk dies, a process crashes, someone ships a bug. A failure is when users notice. A reliable system keeps working correctly even when faults happen, so faults do not become failures.

Faults come from three places: hardware (disks and power fail constantly at scale), software (bugs that appear only under rare conditions), and people (a wrong configuration is a very common cause). You cannot prevent every fault, so you design so that no single fault is fatal.

Availability is the share of time a system is usable. It is usually quoted in nines.

• 99 percent: about 3.65 days of downtime per year.
• 99.9 percent (three nines): about 8.8 hours per year.
• 99.99 percent (four nines): about 53 minutes per year.
• 99.999 percent (five nines): about 5 minutes per year.

Each extra nine is ten times harder and much more expensive. Reliability and availability are related but different: a system that fails for one second every hour is available (99.97 percent) but not very reliable.

Components in a chain multiply. Two parts that are each 99.9 percent available, used one after the other, give 0.999 x 0.999, about 99.8 percent. Chains get worse as they get longer.

Redundancy means having spare components so one can fail without stopping the service. Replication is redundancy for data: several copies kept on different machines.

Two spare machines in parallel behave very differently from a chain. Both must fail at once, so the chance is 0.001 x 0.001, and availability becomes about 99.9999 percent. That is the whole reason to run more than one of anything important.

There are two common styles. In active-passive, one machine works and a standby takes over when it dies, which is simple but wastes the standby. In active-active, every machine serves traffic, which uses everything but needs care with shared data. The costs are real: more money, the work of keeping copies in sync, and the risk of copies disagreeing.

Efficiency is how well you use resources and has two faces. Latency is how long one request takes. Throughput is how many requests you finish per second. A design can have great throughput and poor latency, so state which one you care about.

Serviceability is how easy the system is to operate and repair. Availability depends on it directly: availability is roughly MTBF / (MTBF + MTTR), where MTBF is the mean time between failures and MTTR is the mean time to repair. You can improve availability by failing less often or by repairing faster. Good monitoring, simple deploys, and easy rollbacks all shrink repair time.

When a design review asks "is this scalable and reliable?", answer with specifics. Say how load is spread (horizontal scaling and stateless servers), what happens when each part dies (redundancy), how long recovery takes (serviceability), and what the target availability is. Then say what it costs. Every nine and every spare machine is a tradeoff against money and complexity, and a hobby project does not need five nines.