Event Sourcing & CQRS

Store what happened instead of what is: an append-only log of events as the source of truth, state rebuilt by replaying it, optimistic concurrency on stream versions, snapshots, questions about the past, read models that lag and can be rebuilt, schemas that change, and personal data you must forget in a log that never forgets.

An interactive System Design lesson: 19 steps, about 32 minutes, on a live simulation in your browser.

A bank account, acct-1, for Ada. In most systems the database holds one row, balance = 75, and every change overwrites it. Here the event-store holds what happened instead: AccountOpened and MoneyDeposited 100 on day 1, MoneyWithdrawn 30 on day 3, MoneyDeposited 5 on day 4. Four events, appended and never changed. This is event sourcing.

Watch the api handle each command. It has no current balance to look up, so it reads the stream from the start, replays the events to rebuild the account (0, 100, 70), checks the rule, and appends one new event. The balance, 75, is not stored anywhere on the write side: it is a fold over the events.

What you will learn

1. State as a log

   • An account with no balance column: In event sourcing the events are the database; current state is a fold over them, recomputed when needed.
   • Read your own write: Writes go to the log and reads come from projections that trail it: right after a write, a read can be stale.
   • The projection catches up: CQRS splits the model that decides from the models that answer, and the gap between them is the replication lag of your own making.

2. Commands and concurrency

   • Two writers, one stream: An append succeeds only if the stream is still at the version the decision was based on; otherwise reload and decide again.
   • A command that breaks a rule: Commands are requests in the imperative (withdraw); events are facts in the past tense (MoneyWithdrawn), and only accepted commands become events.
   • Drill: the expected version

3. Replay: snapshots and time travel

   • Snapshots: A snapshot is a cached fold at version N; loading becomes snapshot plus the events after N.
   • The next command after a snapshot: With snapshots the cost of a load is bounded by the snapshot interval, not by the age of the account.
   • What was the balance on day 3?: Every past state is a replay of a prefix of the log: temporal queries come free with event sourcing.
   • Drill: events to replay

4. Read models (CQRS)

   • A projection stops: A projection is a consumer with a position in the log: stopped, it falls behind silently; restarted, it catches up from its position.
   • A question nobody planned for: A new read model can be built from the whole history at any time: you never have to know today which questions you will ask tomorrow.
   • Rebuild after a bug fix: Read models are disposable: fix the code, drop the projection, replay the log.

5. Living with history

   • Version 2 of an event: Old events are never rewritten; an upcaster translates them to the current schema each time they are read.
   • Without the upcaster: Every reader of the log must understand every schema version ever written; an upcaster is how you keep that promise in one place.
   • Forgetting in a log that never forgets: To forget a person in an immutable log, encrypt their data with their own key and delete the key.

6. When not to

   • When not to use event sourcing: Event-source the parts of a system whose history is the product; keep everything else in plain tables.

7. Recap & playground

   • Cheat sheet
   • Playground