Message Queues
Decouple services, absorb spikes and retry safely: acknowledgements, visibility timeouts, duplicates and idempotency, dead-letter queues, backpressure, ordering and pub/sub.
An interactive System Design lesson: 21 steps, about 32 minutes, on a live simulation in your browser.
A shop's checkout, the producer, takes an order and then calls the email service to send the confirmation. The call is synchronous: checkout sends the request and waits for the answer before it tells the customer anything.
The email service talks to an outside mail provider and takes 800 ms. So the customer's checkout takes 804 ms, and 800 of them are spent on an email the customer does not need before seeing "Thank you for your order".
What you will learn
The synchronous chain
- Checkout waits for the email: In a synchronous chain the user waits for the slowest step, even when that step's result is not needed to answer them.
- The email service breaks: A synchronous call couples the caller's success to the callee's health at that exact moment; if the callee fails, the work fails with it.
Publish and carry on
- Put the work on a queue: A queue turns "do this now" into "this needs doing": the producer waits only for the broker to store the message, and the work survives until a worker acks it.
- Nine messages, three workers: Competing consumers share one queue and each message goes to exactly one of them, so throughput scales with the number of workers.
Acknowledgements
- A worker dies mid-message: Unacked is not deleted: a message the broker handed out comes back after the visibility timeout unless someone acks it.
- At-most-once: ack first, lose it: Ack before the work and a crash loses the message: at-most-once. Ack after the work and a crash repeats it: at-least-once.
- At-least-once: the duplicate: At-least-once delivery means duplicates: any message can be processed twice, so the consumer must be safe to run twice.
- An idempotent consumer: Exactly-once processing is at-least-once delivery plus an idempotent consumer: the broker may repeat, the consumer makes the repeat harmless.
Poison and dead letters
- A poison message: A dead-letter queue bounds retries: after N failed attempts a message is parked for a human instead of blocking the queue forever.
- Fix, then redrive
Backpressure
- Producers outrun consumers: A queue's depth is its early warning: when it keeps growing, consumers are slower than producers, and a bounded queue pushes that fact back to the producer.
- Add workers to drain it
Ordering
- Three updates to one order: Competing consumers trade order for throughput: a queue hands messages out in order, but parallel workers finish them in any order.
- Partitions keep per-key order: Partition by key to keep order per key: one partition, one consumer, in sequence; different keys still run in parallel, up to the number of partitions.
- Keys and commits in Kafka
Pub/sub and real brokers
- Pub/sub: every group gets a copy: A queue gives each message to one consumer; pub/sub gives each message to every subscribing group, and one consumer per group.
- Drill: SQS dead-letter setting
- Drill: commit after processing
- Drill: exactly once
Recap & playground
- Cheat sheet
- Playground