Skip to content

Scenario: notification fan-out

One event becomes one job per recipient and channel, each channel paced by its own provider and recorded once.

Predict first

Before you read: predict

A box office's "Show cancelled" button runs this handler. It loops over 2,000 ticket holders and, for each, awaits the email provider before writing the in-app notice. On this run the provider refuses every send from the 601st send on; a refused send throws, and the request fails.

When the request fails, what has been delivered?

Code for this question
for (const holder of holders) {  await email.send(holder, notice);  await inApp.write(holder, notice);}return 202;
Pick one

The 30-second version

Never send from the request: record the event, return, and fan out behind it. Give each channel its own queue, throttled to its provider's rate by one limit its workers share. Size retries to the outage you expect, then replay what failed. A send log unique on (event, user, channel) stops repeats.

One event becomes one job per recipient and channel, each channel paced by its own provider and recorded once.

How it works

Step 1 of 10

Numbers first. Recipients × channels is the job count; each provider's accepted rate bounds each channel's pace.

Analogy: A window cleaner's round called off for rain

On the round In the design
the cleaner hands the customer list to three helpers, one each for texts, emails and notes through the door, and is free at once the request returns; one queue per channel (steps 2–3)
the text bundle allows 100 texts an hour however many phones the helpers use one limit per provider, shared by every worker (steps 4–5)
the email account is locked all afternoon: emails wait in drafts, texts and notes still go a down provider backs up only its own queue (steps 3, 6–7)
each helper ticks one line per round, customer and way before sending, so a second pass sends nothing twice and a text never counts as the note the send log's key (steps 8–9)
no knock before 08:00 on the customer's street: early notes wait quiet hours, held not dropped (step 10)

In real terms: one decision, many messages, each paced and remembered on its own.

Breaks down when: a helper who finds the account locked can phone instead; a queued email cannot change channel by itself.

Contrasting case: a password-reset email: one recipient, one channel, sent while the user waits.

Examples, run for real

4.1 ExamHall EXH-3890 → MR !4190: "Results published" at 18:00

The pilot put every email and in-app job for 40,000 candidates on one queue limited to the email provider's 100 sends per second, so in-app notices waited behind emails (EXH-3890). MR !4190 gives each channel its own queue and a send log unique on (exam, candidate, channel) (steps 3–4, 8).

4.1 model: 40,000 candidates × (email, in-app) · email provider 100 sends per second · in-app 2,000 per second · virtual clock from 18:00:00
4.1 EXH-3890 pilot, one queue at 100 jobs per second: in-app notices by 18:01:00 3,000 of 40,000 · last in-app 18:13:20
4.1 MR !4190, one queue per channel: last in-app 18:00:20 · last email 18:06:40
4.1 MR !4190, the admin publishes again at 18:05, send log unique on (exam, candidate, channel): emails 40,000 · in-app 40,000 · extra sends 0

The admin's second press at 18:05 sends nothing: every (exam, candidate, channel) row already exists (step 8).

4.2 Proof domain, ticketing: the SMS provider down for 30 minutes

"Doors open in 30 minutes" goes to 8,000 fans by push and SMS at 18:30; the SMS provider refuses everything until 19:00.

4.2 model: 8,000 fans × (push, SMS) from 18:30:00 · push service 1,000 per second · SMS queue 100 jobs per second, 4 attempts, exponential backoff from 30 s · SMS provider refuses every attempt until 19:00:00
4.2 push queue: last push 18:30:08 · SMS queue: delivered 0 · failed 8,000 · last attempt 18:35:59
4.2 failed set replayed at 19:00:00 through the send log: delivered 8,000 · sent twice 0

Four attempts with backoff from 30 s span under four minutes, far short of a 30-minute outage, so every SMS lands in the failed set; push is untouched in its own queue. The replay at 19:00 sends each once, because the failed jobs never wrote a sent row (steps 6–8).

4.3 The board for "Design a notification service that sends email, push and in-app messages" (worked, not executed)

Typical question Steps that answer it Mechanism owner
"Design a notification service for email, push and in-app." 1–5 this lesson; job states: async-queue-fundamentals-bullmq
"The email provider is down for an hour. What happens?" 3, 6, 7 this lesson; the backoff formula: async-queue-fundamentals-bullmq
"How does a user never get the same notification twice?" 8, 9 this lesson; the dedupe store's transaction: async-idempotent-consumers

Try it

Answer first, then check. Nothing is saved.

Question 1 of 6

A ticketing site's presale alert goes to 12,000 fans by email and by in-app notice: 24,000 jobs, enqueued fan by fan, email first. The email provider accepts 40 sends per second; the in-app service accepts 1,000 per second. Both channels share one queue whose workers take at most 40 jobs per second.

How many in-app notices have been delivered after the first 90 seconds?

Pick one

Question 2 of 6

A ticketing site's send log's unique key is (event_id, user_id), and a send runs only when its log row is new. 'Presale opens' (ev-41) goes to three fans by push and email; each event's jobs run per fan, push first, then email. The producer times out and publishes ev-41 again with new job ids; then 'Venue changed' (ev-42) goes to the same fans the same way.

What do the provider logs show?

Pick one

Question 3 of 6

Your design so far sends each channel through its own queue. The interviewer's follow-up: "The email provider is down from 19:00 to 20:00, and 'Show cancelled' goes to 1,000 fans at 19:00." The email queue starts at most 50 jobs per second.

Which answer gets every email delivered after 20:00 with nobody replaying jobs by hand?

Pick one

Question 4 of 6

At 19:00 a ticketing site sends 'Kick-off moved' to 6,000 fans by push, email and SMS, each channel on its own queue: push at 1,000 jobs per second, email at 100 and SMS at 50. The push service is down from 19:00 to 19:10; the push queue holds its jobs and starts them again at 19:10.

At what time does the last of the 18,000 notices go out?

Pick one

Question 5 of 6

A ticketing app holds each push during the fan's quiet hours, 22:00 to 08:00 on the fan's own clock, and sends it at 08:00 instead. Its check reads the server's clock, which runs on UTC. 'Resale tickets available' goes out at 21:30 UTC on Wednesday 14 October 2026 to five fans: London (UTC+1 that day), Cairo (UTC+3), New York (UTC−4), Tokyo (UTC+9) and São Paulo (UTC−3).

How many of the five get the push inside their own quiet hours?

Pick one

Question 6 of 6

A ticketing site's SMS queue holds 18,000 'Gate change' texts. The SMS provider accepts 100 texts per second for the whole account; three other queues share that account, so this queue's limiter is set to 25 jobs per second. For the evening the team runs three SMS workers on this queue.

How many seconds until the last text is sent?

Pick one

Go deeper

Pro tip
  1. Say the job count and each channel's tail out loud before drawing a box. 40,000 candidates at 18:00 on two channels is 80,000 jobs, and the email channel alone runs 400 seconds; every later follow-up leans on those two numbers. When: the interviewer gives an audience size. Status: RAN on the model.
Source

No item is keyed on a source; every key is executed on a model whose rules these pages state. Behaviour of the named products is valid as of 2026-10-13.

  • Amazon SNS, Fanout Amazon SNS notifications to Amazon SQS queues for asynchronous processing (the curriculum row's source, fan-out to SQS queues): https://docs.aws.amazon.com/sns/latest/dg/sns-sqs-as-subscriber.html (aws-sns-sqs-subscriber.txt). "decouples sending and receiving components—without requiring each component to be available at the same time" (steps 2–3). The row's other SNS page, sns-common-scenarios.html, returned only its title to a non-browser fetch on 2026-10-13 (aws-sns-fanout.txt, 35 characters) and keys nothing.
  • BullMQ, Retrying failing jobs (the curriculum row's source): https://docs.bullmq.io/guide/retrying-failing-jobs (bullmq-retrying.txt). "When a processor throws an exception, the worker will catch it and move the job to the failed set." (step 7); "With exponential backoff, it will retry after 2 ^ (attempts - 1) * delay milliseconds.".
  • BullMQ, Rate limiting: https://docs.bullmq.io/guide/rate-limiting (bullmq-rate-limiting.txt). "The rate limiter is global, so if you have for example 10 workers for one queue with the above settings, still only 10 jobs will be processed by second." (steps 4–5; the rule drain and sharedQueue follow). "Jobs that get rate limited will actually stay in the waiting state." (step 4).

Checked against the tools and docs as of 2026-10-13.