Every design in this series so far has scaled out. More partitions, more replicas, more nodes.

This one scales in.

The fastest exchanges in the world run almost everything — order manager, matching engine, market data publisher — on a single server, sometimes in a single process. Not because they can’t afford more machines, but because the network hop between two machines costs more than the machine does.

That inversion is what makes this chapter worth reading. Everything you’ve been taught about distributing work stops applying when a round trip costs more than your entire latency budget.

Move $1 from wallet A to wallet B. A million times a second.

That’s the whole problem, and it is the best-structured chapter in this series — because rather than presenting one architecture, it walks through four, each one solving what the previous one broke:

1. In-memory sharding — fast, and loses money when a node crashes
2. Distributed transactions — correct, and you can’t explain why a balance is what it is
3. Event sourcing — auditable, and too slow going over the network
4. Distributed event sourcing — fast, reliable, and shardable

Watching a design fail four times is more instructive than seeing the final answer, so that’s how this is written.

Here is the throughput requirement for a payment backend serving an Amazon-sized store:

1,000,000 transactions/day ÷ 100,000 seconds = 10 TPS

Ten transactions per second. Less than the hotel reservation system, which was already the smallest number in this series.

And payments are harder than either. Because the failure mode is not a slow page or a stale metric — it’s charging someone twice, or taking their money and not paying the seller. There is a real person, a real bank statement, and in many jurisdictions a regulator.

“Show me the top 10 players, and tell me where I rank.”

Two sentences. They sound like the same problem and they are not, and the second one is why this design exists.

Finding the top 10 of anything is easy — you keep a small heap. But telling one player they are #4,328,911 out of 25 million, updated in real time as everyone else keeps scoring, is a genuinely hard query. There is no shortcut: to know someone’s rank you have to know how many people are ahead of them, which means knowing about everyone.

Amazon S3 launched in 2006. By 2013 it held two trillion objects; by 2021, over a hundred trillion.

It is the substrate under a remarkable amount of this series — routing tiles, video segments, email attachments, Kafka’s tiered storage. Every time an earlier design said “put it in object storage and forget about it,” this is what it was leaning on.

So the interesting question is: how do you promise eleven nines of durability on hardware that fails constantly?

Email is the oldest system in this series by decades. SMTP was specified in 1982. POP and IMAP followed. Those protocols still carry the world’s mail, and they were designed for an internet of a few thousand machines where you downloaded your messages and the server forgot them.

Now Gmail has over 1.8 billion users.

This chapter is about what happens when you keep the interface and replace everything behind it. And it produces the largest numbers we’ve seen — by a wide margin.

Here is the entire scale of this system:

5,000 hotels · 1 million rooms
240,000 reservations per day
= 3 reservations per second

Three per second. A Raspberry Pi could serve that. Every other design in this series has been about surviving volume — 13 million messages a second, 50,000 clicks a second, 100 petabytes of map tiles. This one has none of that.

And it is arguably the hardest.

Because for the first time, being slightly wrong is not acceptable. A dropped metric leaves a gap in a chart. A duplicated ad click costs someone money and gets fixed at reconciliation. But selling the same hotel room to two people is a real person arriving at midnight to find no room, and there is no batch job that fixes that.

Most systems in this series can lose a little data and survive. A dropped metric leaves a gap on a chart. A missed location update is corrected 30 seconds later.

This one is different, and the difference changes everything: these numbers become invoices.

Ad click aggregation decides how much advertisers pay and how much publishers earn. A 1% error on a billion clicks a day is millions of dollars, in someone’s favour, every month. That single fact is why this design reaches for exactly-once processing — which we spent the last chapter establishing is expensive and usually unnecessary.

Monitoring is the system that tells you every other system is broken. Which means when it fails, it fails at exactly the moment you need it — and it fails silently, because the thing that would have told you is the thing that’s down.

We’re building an internal metrics platform: 1,000 server pools, 100 machines per pool, 100 metrics per machine — roughly 10 million metrics, retained for a year.

A message queue sits between two services so they don’t have to know about each other. The producer writes and moves on; the consumer reads when it’s ready. Neither has to be up when the other is.

That buys you four things: decoupling, independent scaling of each side, availability when one side is down, and asynchronous communication so nobody blocks.

Simple enough to describe in a sentence. The design is not simple at all, and the reason is that we’re going to build the harder version: not just a queue that hands messages over and forgets them, but one that retains everything for two weeks and lets consumers read it again from any point.