Skip to content

Overload protection

Protect useful completions by deciding what enters expensive service before an overloaded queue decides for you.

Predict first

Before you read: predict

Five proctor refreshes arrive before any service completion. The fixture uses a two-slot admission check.

For how many does the backend emit 503 before the virtual clock advances?

Pick one

The 30-second version

Goodput counts successful responses that remain useful. Expired queued requests can obstruct new work. Bound admission, counting active and queued jobs in this fixture, and reject excess before expensive processing. A waiting room holds users outside the protected service. Rate limits constrain arrivals over time; load shedding protects current capacity. S1 S5

Protect useful completions by deciding what enters expensive service before an overloaded queue decides for you.

How it works

Step 1 of 7

Count useful successful responses, not every arrival or completed job; late results may be worthless to callers. S1

Analogy: A climbing gym's entry marshal

Climbers need equipment checks before entering the roped floor. An uncontrolled inspection backlog can crowd the area while people miss their booked session. The marshal admits only the people the inspection area can handle, asks others to wait outside, and turns away arrivals whose session cannot be reached. Moving the queue outside does not extend anyone's booking.

Climbing gym Service mechanism
Equipment inspection area Expensive backend work
Limit on people inside Admission bound
Waiting outside the roped floor External waiting room
Missed booked session Result beyond its useful deadline

In real terms: keep a backlog from spending the capacity intended for useful work. Breaks down when: people can negotiate another booking; a request deadline must not silently restart. Contrasting case: a quiet gym may enforce a member's daily entry allowance even without crowding. Likewise, a client quota and an overload signal answer different questions. S5

The waiting area also needs a policy for expiry and excess demand. It is not an unlimited storage promise. A rejection still consumes some networking and application resources; protection has a finite operating range. Selecting a bound requires testing the actual service, because request costs can differ. This lesson's identical-cost jobs isolate admission decisions; they do not predict a production safe limit. S6

Examples, run for real

4.1 ExamHall: expired refreshes block useful work

EXH-1640, Expired proctor refreshes occupy the service queue. A local HTTP backend uses one virtual worker. Each job consumes one tick; deadlines are absolute and inclusive. The fixture intentionally finishes expired work instead of cancelling it. Actual HTTP requests remain open until their modelled completion; explicit arrival acknowledgements remove races. S2

The low-load run receives two requests at tick zero with deadline two, then two at tick two with deadline four. The overload run receives eight in the first wave and the same later wave. Count useful completions through tick four, including completions at that tick. Both runs have the same worker and observation window.

example useful completions: low=4 unbounded=2 bounded=4

Expired first-wave jobs occupy later worker turns in the unbounded run. This is a collapse in useful completions, even though service continues. It demonstrates wasted FIFO work, not a measurement of CPU thrashing. MR !1840, Bound proctor admission before expensive refresh work, limits total inside to two. Excess arrivals receive early errors; capacity is available when the later wave arrives. S1

4.2 Catalog: waiting does not restart a deadline

A catalog gateway initially holds three requests outside a backend that permits two inside. Every request has the original absolute deadline one. The gateway admits another only after capacity frees and discards waiting entries whose deadlines have passed. Through tick three, only one result is useful. A request admitted exactly at its deadline cannot finish a one-tick job in time; preserving the deadline exposes that lost opportunity.

waiting room admission layout: inside=2 outside=3; absolute-deadline useful=1

The HTTP transport is real; the waiting-room controller and service-cost scheduler are deliberately small executed models. No virtual tick represents measured wall-clock latency. A real waiting room protects the origin by withholding arrivals; its user experience and admission policy require separate design. S3

Try it

Answer first, then check. Nothing is saved.

Question 1 of 5

Six catalog refreshes arrive before the first completion. The handler is if(full) job.reject=true; queue.push(job); start(). Every job takes one tick and has deadline nine.

By tick six, how many expensive jobs have started?

Pick one

Question 2 of 5

Five catalog requests with deadline nine enter an external FIFO waiting room at tick zero. The controller permits one inside the backend and releases the next immediately after each one-tick completion.

Just after the first completion and replacement admission, how many remain outside waiting?

Pick one

Question 3 of 5

Three refreshes arrive at tick zero and two at tick one after one completion. A service has one backend slot. The fixture compares a full-capacity shedder, a two-arrivals-per-tick limiter, and open admission.

How many 503s does the run record through tick one for the capacity shedder?

Pick one

Question 4 of 5

A catalog worker has a three-slot bound. Six requests arrive at tick zero with deadline two; three arrive at tick three with deadline six, after that tick's completion. Deadlines are inclusive. The trace records [useful completions, successful jobs completed after their deadline].

Which tuple does the bounded FIFO candidate record through tick six?

Pick one

Question 5 of 5

Three requests arrive while completion is held. The candidate sets limit=1 and hint="2", then runs if(full){ reply(503,{Retry-After:hint}); return; } enqueue(). After the test collects all responses, it records [503s before the first completion, responses carrying Retry-After, jobs started before that completion].

Which tuple does the candidate produce?

Pick one

Go deeper

Pro tip

Record admitted, rejected and useful-success counts separately during an overload test. One combined error percentage obscures which population a change affected. Keep this diagnostic view alongside the offered-load curve. AWS Builders' Library, Visibility, read 2026-09-30.

Sources

Fetched and read 2026-09-30. Fixture observations are reproducible in mechanisms.mjs; sources justify the general mechanisms, not the authored scenario inputs.

Checked against the tools and docs as of 2026-09-30.