Testing strategy under a time-box

A streaming take-home's task sheet: “Continue watching lists episodes watched for 2 minutes or more and less than 90% of the way through, at most 10 of them, the most recently watched first.” The candidate's code and tests:

// row.test.mjs
const ep = (id, watched, lastAt = 0) => ({ id, watched, length: 1800, lastAt });
it('hides a short view', () => expect(continueWatching([ep('a', 119), ep('b', 121)])).toEqual(['b']));
it('drops a finished episode', () => expect(continueWatching([ep('a', 1620), ep('b', 900)])).toEqual(['b']));
it('keeps ten', () => expect(continueWatching([...'abcdefghijkl'].map((id) => ep(id, 900)))).toEqual([...'abcdefghij']));
it('newest first', () => expect(continueWatching([ep('a', 900, 1), ep('b', 900, 2), ep('c', 900, 3)])).toEqual(['c', 'b', 'a']));

In the review the engineer makes four one-line edits to row.mjs, one at a time, and reruns her tests: (1) >= 120 becomes > 120; (2) < 0.9 becomes <= 0.9; (3) slice(0, 10) becomes slice(0, 11); (4) the sort becomes a.lastAt - b.lastAt.

A 4-hour streaming take-home. At 3 h 05 min the features work and nothing is tested. Her price list: the resume journey test (pick an episode from Continue watching and resume it, through the API), 20 min; the README's “Not tested” section, 10 min; and rule tests at 5 min each, of which she would like 8. She works through the items in one order and submits at the 4-hour mark; anything unfinished then is left out.

A streaming take-home's episode page shows an Up next row filled by a recommendations service; the task sheet gives her a sandbox key for it. Her one journey test opens an episode in the browser and checks the Up next row. In CI it fails now and then: the service answers slowly, or sends different picks.

Her streaming take-home's three Continue watching rules have unit tests at their edges. She now writes the one journey test: pick an episode from the row and press play.

A streaming take-home's task sheet: “Play resumes an episode at its saved position. With 60 seconds or less left after that position, play starts the next episode at 0 instead. A position saved more than 30 days ago is ignored, and the episode starts at 0.” The route createApp(store, clock).play(profileId, episodeId) returns { episodeId, at }; the rules live in resumePoint(saved, episode, now), in seconds. You have 45 minutes: the “Not tested” list takes 5, a rule test 5 and a journey test 15.

The review call for her streaming take-home. The task sheet: “Skip intro shows while at least 5 seconds of the intro remain, and stays hidden for the first 3 seconds after playback resumes.” Her rule tests: shown at the intro's first second; shown with 5 s of the intro left; hidden with 4 s left; hidden 2 s after a resume. Her one journey test resumes an episode mid-intro through the API and reads the button's state. Her player module also exports introEnd(ep), which returns ep.introStart + ep.introLength, with no test. All green. The engineer asks what's missing from her tests.

The task sheet: “When an episode's credits start, the next episode plays 10 seconds later, unless the viewer presses Cancel.” She runs her tests with node:test's mock timers: setTimeout is mocked afresh in each test, and mock.timers.tick(ms) moves the clock forward ms milliseconds.

// autoplay.test.mjs
it('waits ten seconds', () => { let n = 0; startCountdown(() => n++); mock.timers.tick(9_000); expect(n).toBe(0); mock.timers.tick(2_000); expect(n).toBe(1); });
it('Cancel stops it', () => { let n = 0; const t = startCountdown(() => n++); mock.timers.tick(5_000); t.cancel(); mock.timers.tick(10_000); expect(n).toBe(0); });

The engineer makes four one-line edits, one at a time, and reruns her tests: (1) 10_000 becomes 9_000; (2) it becomes 11_000; (3) it becomes 12_000; (4) cancel clears nothing.