Skip to content

Cache layers and where a cache sits

Where a copy lives decides who shares it, how stale it can get, and who can delete it.

Predict first

Before you read: predict

ExamHall's API runs on 6 instances behind a round-robin load balancer. Each instance keeps exam metadata in its own in-process cache with a 30 s TTL. At t = 0 every instance loads ECON-201's metadata, which says the exam ends at 10:30. At t = 5 s an admin extends the exam to 10:45: the request lands on instance 1, which writes the database and deletes the key from its cache. Between t = 6 and t = 11 s, six candidates each load the metadata once, one per instance (2, 3, 4, 5, 6, then 1).

Which end times do they see?

Pick one

The 30-second version

Every cache layer holds its own copy. A delete or purge reaches only the copies it touches, and a purged layer refills from the layer inside it. An in-process cache is one copy per instance, so after a write the other instances serve their old copy until their own TTL ends. A shared cache gives the fleet one copy.

Where a copy lives decides who shares it, how stale it can get, and who can delete it.

How it works

Step 1 of 10

A request meets the layers in order: browser, CDN edge, reverse proxy, in-process cache, distributed cache, database buffers. The first fresh copy answers.

Analogy: An orchestra changes a bar before the concert

The librarian holds the master score. Each audience member has a printed programme; a screen in each foyer shows the programme; each player reads a copy of their part on the stand; the band room has one whiteboard that every player checks at the break.

In the concert hall In the system
an audience member's printed programme, which nobody can recall the browser's private copy (the Browser row above)
a foyer screen: front-of-house can wipe it, then re-types it from the nearest player's part an edge copy, its purge and its refill (steps 3–4)
each player's part on the stand: quickest to read; after the conductor cuts a bar, only the player who went to the librarian has the new part in-process copies per instance (steps 5–6)
the band-room whiteboard: one walk away, one correction reaches every player the distributed cache (step 7)
the librarian's master score the database and its buffers (step 9)

In real terms: each layer is a copy with its own audience, lifetime and delete path; a write reaches only the copies its delete can touch.

Breaks down when: a player who hears a clash stops and asks, while a cache serves its copy without question. Wrong inference: "the other stands get corrected soon, because players notice." In-process copies are corrected only by their TTL or an invalidation message.

Contrasting case: no parts on the stands; every player reads from the librarian's one score. Nobody plays a wrong bar, and everyone crowds the librarian's desk. That is no cache at all: always fresh, all load on the database.

Examples, run for real

4.1 ExamHall: the extension that half the candidates never saw

EXH-641: an admin extended ECON-201, and for up to 30 s most candidates still saw the old end. The metadata was cached in-process on all 6 instances. MR !967 moved it to the shared cache (TTL 30 s, delete on write). Both versions run on a real fleet; the shared cache is the model.

before: in-process cache (TTL 30 s)
  60 reads at t=0: database loads 6 · copies held 6 · reads that crossed to the shared cache 0
  t=12 admin extends ECON-201; the PUT lands on i4, which writes the database and deletes the key it can reach
  t=13..24 i1:10:30 i2:10:30 i3:10:30 i4:10:45 i5:10:30 i6:10:30 i1:10:30 i2:10:30 i3:10:30 i4:10:45 i5:10:30 i6:10:30
  t=31..36 i1:10:45 i2:10:45 i3:10:45 i4:10:45 i5:10:45 i6:10:45
after MR !967: shared cache (TTL 30 s)
  60 reads at t=0: database loads 1 · copies held 1 · reads that crossed to the shared cache 60
  t=12 admin extends ECON-201; the PUT lands on i4, which writes the database and deletes the key it can reach
  t=13..24 i1:10:45 i2:10:45 i3:10:45 i4:10:45 i5:10:45 i6:10:45 i1:10:45 i2:10:45 i3:10:45 i4:10:45 i5:10:45 i6:10:45
  t=31..36 i1:10:45 i2:10:45 i3:10:45 i4:10:45 i5:10:45 i6:10:45

Before: six copies, six loads (step 5); the delete fixed only i4, and the rest caught up when their TTL ended at 30 (step 6). After: one copy and one delete for everyone, paid for with 60 round trips to the shared cache (step 7).

4.2 One page, every layer, in a real browser

Chromium → edge model → origin, with three policies: a hashed image (public, max-age=31536000, immutable), exam metadata (public, max-age=0, s-maxage=30) and an attempt (private, no-store). Two candidates use separate browser contexts. Then the metadata changes, the image is replaced at the same URL (the mistake hashed URLs prevent), and both are purged at the edge.

candidate A  GET /media/q7.3f9a.png     edge MISS -> origin  v1 (origin #1)
candidate A  GET /media/q7.3f9a.png     browser cache        v1 (origin #1)
candidate A  GET /exams/econ-201/meta   edge MISS -> origin  ends 10:30 (origin #1)
candidate A  GET /exams/econ-201/meta   edge HIT             ends 10:30 (origin #1)
candidate A  GET /attempts/a-7          edge PASS -> origin  answers of a-7 (origin #1)
candidate A  GET /attempts/a-7          edge PASS -> origin  answers of a-7 (origin #2)
candidate B  GET /media/q7.3f9a.png     edge HIT             v1 (origin #1)
candidate B  GET /exams/econ-201/meta   edge HIT             ends 10:30 (origin #1)
candidate B  GET /attempts/b-3          edge PASS -> origin  answers of b-3 (origin #1)
-- admin edits: meta -> ends 10:45; q7 image replaced at the SAME URL; both paths purged at the edge
candidate A  GET /exams/econ-201/meta   edge MISS -> origin  ends 10:45 (origin #2)
candidate A  GET /media/q7.3f9a.png     browser cache        v1 (origin #1)
candidate C  GET /media/q7.3f9a.png     edge MISS -> origin  v2 (origin #2)
origin requests: /media/q7.3f9a.png 2 · /exams/econ-201/meta 2 · /attempts/a-7 2 · /attempts/b-3 1

A's second image request never leaves the browser; B gets A's edge copy (steps 1–2). The attempt is never stored at the edge (private; see sd-cdn-edge-caching). After the purge, A gets the new metadata but keeps the old image, while C gets the new one: the purge cannot reach A's browser.

Picture it

a man at a small workbench works with a folding pocket tool. Behind him stands a tall two-door cabinet, and through a large window a warehouse with loading bays is visible in the distance: the nearest copy, a closer store, and the far source.
An illustration to help it stick. The mechanism is in the sections above.

Try it

Answer first, then check. Nothing is saved.

Question 1 of 5

A product API runs 4 instances behind a round-robin load balancer. Each keeps products in its own in-process cache with a 1-hour TTL, so nothing expires during the test. 400 requests arrive one after another; request i asks for product i mod 25, so all 25 products are asked for 16 times each.

How many times does the database load a product?

Pick one

Question 2 of 5

A catalogue API runs 3 instances, each with its own in-process cache (TTL 60 s). A CDN in front stores /titles/42 for 300 s, and browsers check the CDN on every view. The mobile app calls the API directly, so each instance loaded title 42 at t = 50 s. At t = 100 s an editor fixes the title's name: the write lands on instance 1, which updates the database and deletes its own copy, and the team purges /titles/42 at the CDN. The next web view, at t = 101 s, misses at the CDN and reaches instance 2.

From when do web views show the fixed name?

Pick one

Question 3 of 5

An event's start time is read through 6 instances, one read a second. At t = 30 s the organiser moves the start from 19:30 to 20:00. Two goals: every read after the edit shows 20:00, and the database serves the event's row at most 2 times in the minute.

Which placement meets both?

Pick one

Question 4 of 5

An event page reads the start time through two tiers: each of 4 instances keeps an in-process copy for 10 s in front of a shared cache with a 60 s TTL. Every instance runs this on an edit:

After an organiser moves a start time, most fans keep seeing the old time for several seconds, then all see the new one.

Which is the bug?

Code for this question
async function updateEvent(id, patch) {  await db.events.update(id, patch);   // the database is the source of truth  await shared.del(`event:${id}`);     // the shared cache (one copy for the fleet)  local.delete(`event:${id}`);         // this instance's in-process copy}
Pick one

Question 5 of 5

A streaming app's API runs 3 pods behind a round-robin balancer, 6 requests a second. Every request reads the plan-price table from the shared cache (TTL 60 s), a network round trip each time. Prices change only in a release, which updates the database and restarts every pod. The team wants at most one round trip per pod per release, and no request ever served an older release's prices.

Which change does that?

Pick one

Go deeper

Pro tips
  1. Stamp every response with the instance and the layer that answered it (x-served-by: i3, x-cache-source: local|shared|db), then read the same key in a loop through the load balancer. When: a bug report says "some users see the old value" (steps 5–6). Verify: the script's i2:10:30 … i1:10:45 line is exactly this loop; the divergence shows in one line.
  2. Give every in-process entry a maximum TTL, even when you send invalidation messages. A lost message then leaves a copy stale for at most that TTL, not forever. When: adding a local tier in front of Redis (step 8). Status: DOC. Redis's client-side caching reference advises that "Putting a max TTL on every key is a good idea", and says to flush the local cache when the invalidation connection is lost. Not run here (no Redis).
Source
  • MDN, HTTP caching: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Caching · fetched 2026-09-27. "Types of caches": private caches, shared caches, and managed caches such as reverse proxies and CDNs (steps 1, 4). "Deleting stored responses": once a response is stored with a long max-age, "there is nothing the server can do", because no more requests reach it (step 3). Valid as of 2026-09-27.
  • Redis docs, Client-side caching reference: https://redis.io/docs/latest/develop/reference/client-side-caching/ · fetched 2026-09-27. Local caches in application memory cut latency and database load; the problem is invalidation, since the application "may continue to serve the old username" after a change; tracking sends invalidation messages to clients that may hold the key. Valid as of 2026-09-27.
  • PostgreSQL 18 docs, Resource consumption, shared_buffers: https://www.postgresql.org/docs/current/runtime-config-resource.html · fetched 2026-09-27. "the amount of memory the database server uses for shared memory buffers"; PostgreSQL "also relies on the operating system cache" (step 9). That a buffer hit still needs a connection and a query is this lesson's reasoning, not a quote. Valid as of 2026-09-27.

Related lessons

Checked against the tools and docs as of 2026-09-27.