sebastienrousseau/agtmls/verification-before-completion

Verification before completion

Trigger. You are about to write done, fixed, working, should work, ready, that's it, all set, or to mark a task complete — or a human asked whether something is finished.

Reference material (the full claim→evidence table, the rationalization

catalogue with counter-moves, and worked pass/fail transcripts): see

reference.md in this directory. This file is the gate and the loop.

The rule

**A claim you have not observed is a guess. Report it as one, or go and

observe it.**

This is not about being slow or hedging. It is about the difference between I ran it and saw X and it should do X. The first is a report. The second is a prediction wearing a report's clothes, and it is the single most common way agent work goes wrong: not a bad edit, but a correct-looking edit reported as verified when nobody looked.

The loop

Run this before every completion claim. It is four questions, not four commands: often the answer to step 2 is one observation you already have.

1. What exactly am I claiming?

Write the claim as one sentence, in the form a reader will act on. Vague claims cannot be verified, so sharpen first:

2. What observation would justify it?

For each claim, name the artifact that proves it — a command's exit code and output, a file's content, a screenshot, a log line. If you cannot name one, the claim is not verifiable as stated: go back to step 1.

The mapping is mechanical. The full table is in reference.md; the load-bearing rows:

3. Did I actually observe it — in this state of the tree?

The trap is stale evidence: a green run from three edits ago, a test seen passing before the last refactor, output from a different branch. Evidence expires the moment the tree changes.

Ask literally: did I run this, after my last edit? If the answer is "I ran something like it earlier", the answer is no.

4. What did I not check?

Name it explicitly in the report. An honest gap is information; a silent gap is a defect you handed to someone else. See Reporting below.

Size the check to the risk

Checking more than the claim needs is its own failure. It burns time and tokens, and a heavy verification ritual on every step is a documented way skills make agents worse. The rule governs what you claim, not how much you run:

• Claim what the risk calls for. A typo or comment fix needs "the text

now reads X": one look, not "nothing else broke". A change inside one function needs its targeted test, failing first if it fixes a bug. Claim "nothing else broke", and run the full suite for it, when the change reaches shared code, or before a merge or release; that is usually the project's own gate.

• Irreversible or outward-facing work gets the most. Releases,

migrations, deletions and security fixes: the full gate, then read the result back from where it landed.

• One observation per claim. Once you have it for this tree, stop. Do

not re-run an unchanged check to feel sure.

• Do not loop. If a check fails twice for reasons outside the change

(no network, a missing service, a flaky harness), report the claim unverified with the command to run, instead of retrying.

• Never act just to verify. Do not deploy, push, send or delete to prove

a claim. Ask, or report it unverified.

• Downgrade instead of over-running. When the evidence would cost more

than the claim is worth, make the narrower claim and name the gap.

Red flags — none of these is verification

Each of these has been mistaken for evidence. None of them is:

• "It compiles" / "it typechecks." Necessary, never sufficient. The

compiler proves shape, not behaviour.

• "It looks right." Reading a diff is not running it. You are checking

your own intent against itself.

• "The test passes." Did you see it fail first? A test that never failed

proves the harness runs, not that the behaviour is pinned.

• "Green on my machine." Not the full gate. Partial runs hide

cross-module breakage.

• "The linter is wrong here." Maybe. Justify a specific, inline

suppression, never a blanket one.

• A delta inside noise. A 3% benchmark change with 8% run-to-run variance

is not an improvement. State the variance or make no claim.

• The tool reported success. Tools report their success. git push

succeeding says nothing about CI.

Rationalizations to reject

These are the sentences that talk you past the gate. Each has a counter-move; the full catalogue with worked examples is in reference.md.

That last row is the important one. Being unable to verify is an acceptable outcome. Claiming verification you do not have is not.

Reporting

State what you did, what you observed, and what you skipped. Three shapes:

Verified. Say it plainly, with the observation. No hedging — hedging a verified result is its own failure mode.

Fixed. pytest tests/test_parser.py now passes (14 passed); it failed with

KeyError: 'root' before the change. Full suite: 214 passed, 0 failed.

Partly verified. Lead with what holds, then the gap.

The parser change works — the failing input now returns 3, confirmed by the

new test. I have not run the integration suite; it needs a live database I

can't reach from here.

Unverified. Say so first, before the description.

Unverified: I could not run this. The change follows the pattern in

handlers/auth.py:88 and should make the timeout configurable, but I have

not executed it. Run make test-integration to confirm.

When not to use

• The project defines its own evidence bar, gates, or test taxonomy — follow

that first; this skill is the general habit underneath it.

• Pure discussion, brainstorming, or exploration with no completion claim.
• You are debugging and do not yet have a fix to verify — that is

systematic-debugging.