Amazon S3 launched in 2006. By 2013 it held two trillion objects; by 2021, over a hundred trillion.

It is the substrate under a remarkable amount of this series — routing tiles, video segments, email attachments, Kafka’s tiered storage. Every time an earlier design said “put it in object storage and forget about it,” this is what it was leaning on.

So the interesting question is: how do you promise eleven nines of durability on hardware that fails constantly?

Email is the oldest system in this series by decades. SMTP was specified in 1982. POP and IMAP followed. Those protocols still carry the world’s mail, and they were designed for an internet of a few thousand machines where you downloaded your messages and the server forgot them.

Now Gmail has over 1.8 billion users.

This chapter is about what happens when you keep the interface and replace everything behind it. And it produces the largest numbers we’ve seen — by a wide margin.

A message queue sits between two services so they don’t have to know about each other. The producer writes and moves on; the consumer reads when it’s ready. Neither has to be up when the other is.

That buys you four things: decoupling, independent scaling of each side, availability when one side is down, and asynchronous communication so nobody blocks.

Simple enough to describe in a sentence. The design is not simple at all, and the reason is that we’re going to build the harder version: not just a queue that hands messages over and forgets them, but one that retains everything for two weeks and lets consumers read it again from any point.

You have a 4 GB video file in your Drive folder. You change its title — a few dozen bytes near the start.

How many bytes should cross the network?

The naive answer is 4 GB. The right answer is a few hundred kilobytes, and getting from one to the other is what this chapter is about. Everything else — the API servers, the metadata database, the notification service — is machinery you have already seen. The distinctive problems here are three: