CDNs
Serve bytes from the edge, close to every user: what distance costs, what an edge saves the origin, how Cache-Control and versioned file names decide what users see after a deploy, and what happens when an edge or the origin goes down.
An interactive System Design lesson: 20 steps, about 30 minutes, on a live simulation in your browser.
A shop runs one web server, the origin, in a data centre in us-east. Its customers are everywhere: user-us in New York, user-eu in Frankfurt, user-asia in Singapore. Each asks the origin for the home page, /index.html.
The origin spends the same 40 ms on each of them. What differs is the wire. user-us is 30 ms away and gets its page in 100 ms. user-eu is 90 ms away and waits 220 ms. user-asia is 160 ms away and waits 360 ms: more than three times as long, for identical work.
What you will learn
What distance costs
- One origin, three continents: Latency is distance times two plus work. You can buy faster servers; you cannot buy a shorter ocean.
- An infinitely fast origin
An edge near every user
- An edge near each user: An edge is a cache that lives in a city. The first request in a region pays the full trip to the origin; everyone after it pays only the trip to the edge.
- Who pays the trip?
- What the origin sees now: A CDN changes the origin's load from the number of requests to the number of distinct files times the number of edges, once per TTL.
Cache-Control
- The origin decides, in a header: A CDN caches what the origin's headers allow for as long as they allow. The TTL is a promise you make about how stale you can afford to be.
- Drill: cache a build artefact forever
Shipping a new version
- Ship v2, users get v1: An edge serves what it has until the TTL says otherwise. A deploy at the origin changes nothing at the edge.
- When does Europe get v2?
- Purge: tell every edge to forget
- Versioned file names: Never change a cached file: give the new content a new name. Only the small HTML that points to the names needs a short TTL.
When an edge or the origin dies
- The Frankfurt edge goes down: When an edge dies its users fail over to the next nearest edge. They stay up and pay that edge's distance, plus a miss for anything it has not cached.
- The origin goes down: During an origin outage a CDN serves what each edge already holds and nothing else. Coverage is per edge, not per file.
- Serving stale on purpose
What a CDN cannot cache
- Writes cross the ocean anyway: A CDN speeds up what is the same for many users and changes rarely. Writes and per-request answers still travel to the origin at full distance.
- Somebody else's account page: An edge's cache key is the URL, not the person. Anything that differs per user must say private or no-store, or it will be served to the wrong user.
- Drill: keep it out of every cache
- Hit ratio with a long tail: A CDN's hit ratio is set by the shape of the traffic: the popular head is nearly free, the long tail keeps the origin busy.
Recap & playground
- Cheat sheet
- Playground