What to Read Next: Engineering Blogs, Papers, and How to Read Them
Abhay 13 min read

What to Read Next: Engineering Blogs, Papers, and How to Read Them

Closing
Listen to this article
Read aloud in your browser

Twenty-eight designs down. So — what now?

The usual answer is a list of engineering blogs. Here is one, further down the page. But a list on its own will not make you better, and it is worth being honest about why.

Try this. Think of the last engineering post you read. Can you say what problem the team had, what number it broke at, and what the fix cost them?

Most people can’t. What survives is a brand and a technology: “Discord uses ScyllaDB.” That is the one detail in the whole article you will never be able to use.

So let’s start with how to read, and then get to what.

How to read an engineering blog post

Every good post is telling the same four-part story, whether or not the author lays it out that way. Your job is to pull those four parts out.

flowchart LR
    P["1 · Pressure
What grew until
something broke?"]
    F["2 · Failure
What broke, and
at what number?"]
    R["3 · Response
What changed, and
what did it cost?"]
    G["4 · Generalisation
Where else does this
shape show up?"]

    P --> F --> R --> G

    style P fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e
    style F fill:#fee2e2,stroke:#dc2626,color:#7f1d1d
    style R fill:#dcfce7,stroke:#16a34a,color:#14532d
    style G fill:#fef3c7,stroke:#d97706,color:#78350f

Step 4 is the whole point. Steps 1–3 are about someone else’s system. Step 4 is about yours.

Worked example: Discord, twice

Discord has written about the same database table twice, six years apart. Two posts, two completely different problems — which makes them a perfect pair to practise on.

The 2017 post — billions of messages, on Cassandra.

Pressure. A channel collects messages forever. Its Cassandra partition just keeps growing. Failure. Huge partitions choke garbage collection during compaction. And, as the post puts it, “the data in it cannot be distributed around the cluster.” Response. Put time in the partition key: (channel_id, bucket), with roughly 10-day buckets. Now no partition can exceed about 100 MB, by construction. Generalisation. Any partition key built from something that accumulates forever, with nothing to bound it, will eventually blow up.

That last line is the part you keep. It has nothing to do with Cassandra — it applies just as well to Kafka topics, DynamoDB, or a sharded MySQL cluster. One post, read this way, hands you a lens you will reuse for years.

The 2023 post — trillions of messages, moving to ScyllaDB.

Here is the twist, and it is a good one. Bucketing had capped how big a partition could get. It did nothing about how busy one could get.

A channel on a server with hundreds of thousands of members gets orders of magnitude more reads than a channel among five friends. Those partitions ran hot no matter how small they were. And the fix was not a cleverer key — it was a Rust service in front of the database that collapses simultaneous requests for the same row into a single query.

Generalisation. Capping the size of a partition says nothing about capping the load on it. Two different problems — and solving the first one can hide the second.

Three habits that make this stick

Read the numbers, skip the adjectives. “Massive scale” tells you nothing. “177 nodes down to 72, p99 for old messages from 40–125 ms to 15 ms” tells you everything: how big this system is, and whether yours is anywhere near it. Most posts describe systems a hundred times larger than yours, and their solution is often wrong for you.

Hunt for the sentence where they admit a cost. “This increased write amplification.” “We accepted eventual consistency here.” Every honest post has one, and it is where the real engineering is hiding. A post without one is a recruiting ad.

Ask what they turned down. The options a team rejected tell you more about their constraints than the one they picked. Good posts say so outright. For the rest, work it out yourself — that is the same muscle an interviewer is testing.

In a hurry? Read these three

The rest of this page is long. If you close the tab after this section, take these:

WhatWhy
1The Dynamo paper (an evening)Almost everything in Chapter 6 comes from here. Read it and half the field clicks into place.
2Designing Data-Intensive Applications (a few months)The single best book in this space. Nothing else is close.
3Cloudflare’s blog (weekly)The most consistently excellent engineering writing published anywhere right now.

Everything below is elaboration on those three.

Papers worth your evenings

A paper is slower going than a blog post and worth far more. Blog posts get rewritten, migrated and quietly deleted; papers sit still for twenty years.

Each of these introduced something you have already met in this series — which means you are reading them with the answers in hand, and that is much easier than it sounds.

Architecture write-ups

Want the shape of a whole system rather than one mechanism? These are the ones to read.

High Scalability catalogued production architectures for over a decade. It no longer publishes, but the archive is one of the best surveys of how large systems were actually assembled — and unusually, it covers the unglamorous decisions.

Talks

Sometimes an engineer explaining their own system out loud beats anything written down.

Company engineering blogs

Company blogs are recruiting instruments as much as technical ones, so they vary enormously. These are the ones that consistently publish something with a number in it.

Consistently excellent

BlogWhy it earns the subscription
CloudflareUnmatched on networking, DDoS, TLS and edge compute. Publishes real postmortems with real numbers.
Meta EngineeringSuccessor to code.facebook.com. Storage, ML infrastructure, and the largest-scale problems anyone writes about publicly.
Netflix TechBlogStreaming, chaos engineering, personalisation. The origin of a great deal of standard practice.
Uber EngineeringReal-time geospatial systems, and the most candid migration write-ups in the industry.
DiscordRare and specific: millions of concurrent WebSocket connections, described honestly.
Dropbox TechSync, storage, and the famous move off S3. Directly relevant to Chapter 15.
StripeIdempotency, correctness under partial failure, API design as a discipline.

Strong and current

BlogFocus
GitHubGit at scale, MySQL, availability
ShopifyFlash-sale traffic spikes, Ruby at scale, sharding
InstacartLogistics, search, ML systems
PinterestRecommendations, storage, home feed
GrabReal-time systems in Southeast Asia; excellent on geo
AirbnbSearch, payments, data infrastructure
YelpSearch ranking, data pipelines
SpotifyEvent delivery, ML, developer platforms
SlackReal-time messaging, mobile sync
LinkedInKafka’s birthplace; graph and feed systems
CanvaNewer, unusually concrete on media processing
AWS ArchitectureReference patterns and well-argued trade-offs

Individuals, who are often better than the companies

Company blogs are recruiting instruments with engineering content attached. These are just engineers writing.

  • Marc Brooker (AWS) — short essays on distributed systems that are frequently better than the papers they discuss. Start with anything on timeouts or retries.
  • Werner Vogels — All Things Distributed — Amazon’s CTO, writing since 2004.
  • Dan Luu — empirical, contrarian, heavily footnoted. His work on latency and on the actual cost of complexity will change decisions you make.
  • Jepsen — Kyle Kingsbury breaking distributed databases and documenting exactly how. Read one analysis of a database you use; it is a bracing experience.
  • Murat Demirbas — a distributed systems researcher’s paper reviews. The fastest way to decide whether a paper is worth your evening.
  • The Pragmatic Engineer — Gergely Orosz on how engineering organisations actually operate. Adjacent to system design, and the adjacency matters.

Books, courses and the deep end

Blogs tell you what one company did on one Tuesday. These tell you why any of it works.

Books

  • Designing Data-Intensive Applications — Martin Kleppmann. If you read exactly one book after this series, read this one. It supplies the theory every chapter here assumed: replication, partitioning, transactions, consensus, and the failure modes underneath.
  • Google SRE books — free online. Site Reliability Engineering and The SRE Workbook cover the half of system design that interviews ignore and production does not: SLOs, error budgets, on-call, and what a real postmortem contains.

Courses

Papers, ongoing

Aggregators

Turning reading into competence

It is entirely possible to read constantly and learn nothing. Plenty of people manage it. What separates them from everyone else is what happens after the tab closes.

flowchart TD
    A["Read one post per week
from a feed you subscribed to"]
    B["Pull out the four parts:
pressure, failure,
response, generalisation"]
    C["Write 3 sentences
in your own words"]
    D["Ask: which system I own
has this same shape?"]
    E["Re-derive the design
from scratch, a month later"]
    F["You now own the idea"]

    A --> B --> C --> D --> E --> F
    E -.->|"can't do it?
you never understood it"| A

    style A fill:#e0f2fe,stroke:#0284c7,color:#0c4a6e
    style C fill:#fef3c7,stroke:#d97706,color:#78350f
    style E fill:#dcfce7,stroke:#16a34a,color:#14532d
    style F fill:#dcfce7,stroke:#16a34a,color:#14532d

That re-derivation step is the one everybody skips. It is also the only one that proves anything.

Here is the test. A month after reading the Dynamo paper, close the tab and explain why hinted handoff exists and what breaks without it. If you can’t, you do not know the paper — you remember having read it. Those feel identical from the inside and are worth wildly different amounts.

A concrete cadence that works:

FrequencyPractice
WeeklyOne blog post, read for the four parts. Three sentences written down.
MonthlyOne paper, slowly. Keep Kleppmann next to you as a decoder ring.
QuarterlyRe-derive one design from memory, then compare against the original.
Whenever you’re stuckHit a problem at work? Go find who has already written about it. Someone has.

That last row is worth more than the other three combined. Reading about consistent hashing on a quiet Sunday is mildly interesting. Reading about it the week you are staring at one shard carrying 40% of your traffic is how it becomes permanent.

One last thing

Looking back over twenty-eight designs, the striking thing is how few ideas there actually were.

Partition to distribute load. Replicate to survive failure. Cache to avoid repeating work. Queue to decouple producers from consumers. Denormalise to trade write cost for read speed. Batch to amortise. Version to detect conflict.

Seven ideas. That is very nearly the whole vocabulary.

YouTube, Google Drive, a news feed and a chat system are all the same seven, mixed in different proportions under different pressures. A rate limiter and a URL shortener look like completely different problems. They are the same problem in different clothes: bounded state, enormous read volume, and enough tolerance for approximation to get away with it.

This is why reading compounds. Each new architecture is not another thing to memorise — it is one more data point on how those same seven behave under a pressure you had not seen before.

Around the twentieth post, something shifts. You stop reading them as stories and start reading them as variations. That is when system design stops being an interview topic and starts being a way of looking at things.

The list above is a starting point, and starting points go stale. The method is what lasts.

References and Further Reading

Papers

Books and courses

Ongoing sources

In this series