joetawil7/first-pass/ship-check

ship-check

Walk every step. A step you cannot do goes in the report under "Not verified" with the reason; it is never skipped silently. The repo's commands, test limits and heavy-run rules are in the first-pass:project block of its instruction file (AGENTS.md or CLAUDE.md); if there is none, read them from the CI config and say so in the report. Its test limits bind every step below, and so do the machine block's limits (how this machine bounds a heavy run, and the browsers and devices it can look at a UI with).

Run everything inside the repo that changed (cd <repo>, git -C <repo>), not from a main folder above it. A change that spans repos walks the steps once per repo.

A prompt with several items walks steps 1 and 2 per item, while each is built, running only the tests that item touches; one clean checkout of the base serves every item's fail-first run. Steps 3 and 4 run once for all of them, after the last item is built: one review per item (small items that touch the same code can share one), started together only where the repo's test limits say side-by-side runs are safe (at most three at once), otherwise one after another; and, while they run, CI's full checks in one clean checkout holding every item. A later edit reruns only the checks it can affect, and the full checks run once more on the final change (step 3). The report answers each item, and every item's list of smaller findings comes in it, once.

Nothing is pushed, merged, deployed, migrated or published until steps 3 and 4 are finished for a clean checkout holding exactly what it ships (failures the base has too are named, and findings left open are answered by the user first), unless the user says to ship it as it is.

0. Size

A change with no logic in it (a comment, a doc, a spelling fix that changes no behaviour) runs only the repo's format, lint and build checks, plus step 6 when a person reads the text, and the report says this exception was used. Anything else, however small (a constant, a condition, a default, a price, a label whose meaning changes), walks every step. When unsure, it is not a typo.

1. Pre-mortem

Find the pre-mortem in the plan. If there is none, write it now from the diff with the premortem skill, and say in the report that it was written after the code. Search again for neighbors of every field, status, option, queue and endpoint in the diff; add any the plan missed.

2. Tests that fail on the old code

For each behaviour the change adds or fixes, and each pre-mortem answer of the "test" kind:

1. Right layer. The lowest layer that reproduces it end to end. If the bug could live

in a query, a transaction, a queue or a browser, the test runs against the real thing (an integration test on a real database, an end-to-end test in a browser). A unit test with mocks is enough only for pure logic.

1. Fails first. In a clean checkout of the code before the change. That is the base:

HEAD when the change is still uncommitted, otherwise the commit the branch started from (git merge-base HEAD origin/<main branch>). Never a checkout that already contains the change, or good tests pass on the "old" code and look worthless. `` git -C <repo> worktree add --detach "<temp dir>/<repo>-shipcheck-<short base sha>-<time>" <base> `` Put it in the system temp folder, not beside the repo (a new folder inside a main folder looks like a new repo to every tool that scans it), under a name no parallel session will pick. Copy in only the new or changed test files, install dependencies as CI does, run them, and record which fail and why. A run here that could reach real people follows step 4's first paragraph, env files included. Then copy in the change and record that they pass. A test that passes on the old code proves nothing about the change: rewrite it.

1. Guards (tests that behaviour which must not change still holds) pass on both. Say

which tests are guards.

1. The risky answers get a test. When the change has a queue job, a paid call, a

publish, a payment or a delete in its path: one test runs it twice at once, one makes the outside call time out or return a 5xx.

1. Tests assert what the user sees or what is stored, not only that a new test id

exists or that a mock was called.

1. Look at the UI running. A change to what a screen shows or how it behaves is opened

with the repo section's UI check and the browsers and devices the machine block names: the web at a desktop and a phone width, a native app on each device class it ships to, in the states the change touches. Save the screenshots outside the repo and list their paths. With no way on this machine, say "Not checked: the UI was not looked at running", why, and what to install to look (the machine block's "Missing:" line, or the tool's current docs); install it only with the user's yes. A run here that could reach real people (a dev server, an app, or a UI check that may reuse a server already running) follows step 4's first paragraph.

3. Fresh review

Hand the change to the breaker agent (Claude Code: first-pass:breaker from the plugin, or breaker where a repo installed its own; Cursor: /breaker) with what the change is for, the repo, the base ref or file list, the pre-mortem, and the task's scope (or that the user lifted it with hulk). It must run in its own context. If your tool cannot start one, ask the user to run the breaker in a new chat; never review in the context that wrote the code.

Start it in the background and don't wait for it: while it runs, do what reads or runs but does not edit the files under review: steps 4 to 7 (CI's checks in the clean checkout, monitoring, words, invariants) and a draft of the report. Edits wait for the review (a fix found meanwhile joins its fixes), so the reviewer never reads a tree that is changing. Only runs the repo's test limits and the machine block's limit on heavy runs at once allow beside the review go at the same time, counting the reviewer's own: tell the reviewer which ports, databases and suites the session will use while it runs, so it leaves them alone, and how many heavy runs it may start meanwhile. Until the review's result is in and handled, every reply says built, not done, and what it waits on; nothing of the change is committed to the user's branch, pushed or merged, unless the user says to ship it as it is (commits in a temp clone made for a review are fine); and either way it is not called done. Any edit CI's clean checkout does not hold (a review-led fix, a fix the user picked, or one a check led to) is copied into it once any run there has finished or been stopped, and reruns only the checks it can affect: the tests of the files it changed, the unit, integration and end-to-end (browser) test files that reach them (directly or through their callers), the part that failed when a check led to it, and the format, lint, type and build checks of the packages it touched. A part it reaches through shared code runs whole only when the edit changes what that other code sees (a shared package's exports, a migration, a config value, a route's or event's shape). A test run that selects no tests (the test-list steps a --no-tests run skips do not count as one) is not a pass: fix the selection, or run that part whole. The words, monitoring and invariants checks run for what it changed. CI's full checks (step 4) run once more, in the clean checkout holding the final change, unless every part of them has already run whole, from start to end, after the final change was copied in, once no review or fix is pending and nothing is left for the user to pick or answer, the user picks nothing more from a list, or the user asks to call it done, commit, push or ship; a report that ends with a list or a question (step 8) goes out with them still to run, and says how long they take. Nothing is called done before they pass (failures the base has too are named), and nothing is committed, pushed or shipped before them unless the user says to ship it as it is; a later edit with no logic in it needs only what step 0 says. Until they pass, write "full checks still to run" in the task list, and every report says built, not done, and which checks ran on the latest edit. When the result is in, a confirmed finding you list rather than fix that breaks an invariant goes into its Known breaks (step 7).

Sort each finding by its worst case, never by the severity the reviewer gave it:

• Real harm: fix it now. Real (CONFIRMED, or PLAUSIBLE and you confirm it), and its

scenario does real harm (money lost, wrongly charged or spent without a cap, lost or leaked data, a side effect done twice, sent wrong or sent without the yes it needs, a security hole, a legal breach, a crash, work left stuck: a job that never finishes, or a person who can't finish what they started), or it stops the change doing what it was for. Fix it with its own failing-first test (step 2), and run the breaker on that fix, again on each new real-harm fix, until one review finds no new real harm. From the second review on, real harm whose Who meets it is unusual waits instead: collect it, fix it in one batch once the item's other fixes are reviewed (each with its own failing-first test), and give the batch one review. Rare-path harm that review, or any later review of the same item, finds starts no new round: it goes first on the list (step 8) as real harm; nothing of the item ships until the user answers it, and while it stays unfixed the item is built, not done. Harm met in normal use (everyone, feature) is still fixed as it is found. Write each finding that waits in the batch, or for the user's answer, in the task list when it is found, and say after each review which harm waits, so a compacted context cannot lose it.

• Smaller: list it. Real, but none of the above: words wrong in some state, a button that

shows when it does nothing, a clumsier path. Don't fix it, and don't ask the user about it during the work: put it on the item's list with its worst case, who meets it and what fixing it would change.

• Disagree: say why in the report, with file:line.
• Older (inside the task's scope, but the change neither caused it nor made it worse): on

the list, marked as older than the change, whatever its harm; a real harm among them is named first.

• Outside the task's scope (the rules' "Stay in the task's scope"): not a finding. When

it does real harm, one line in the report under "Outside this task" (and a Known breaks line when it breaks an invariant, step 7); otherwise left out. A problem the change caused or made worse is never outside, wherever it lives.

A fix that touches code another item in the same prompt uses gets its review on the combined diff. Say after each review what it found and what is left ("item 1: review of the fix found no new real harm; 3 on the list"), so it survives a compacted context.

The list goes to the user once, at the end (step 8), numbered, each with its worst case, who meets it and what fixing it would change, with one question: which to fix. The fixes they pick are built together, with the proof step 2 asks for (or the one the Jev judge picks, below), and get one review together. That review's real harm is fixed as above; its smaller findings go on a new list in that report, not fixed unasked.