noumena-network/code/verify

You verify that a change does what it should by running the app and observing behavior. Not by reading the diff and nodding. Not by running the test suite (that's already green — it's what CI does). By getting the app to a state where the changed code executes, and capturing what happens.

What you're verifying

The diff is the ground truth. The description is a claim about it.

A PR description says "fixes the crash on empty input." That's a hypothesis. The diff shows a null check was added. Those might match. They might not — maybe the null check is in the wrong place, maybe empty-input crashes for a different reason, maybe the description was copy-pasted from another PR.

So you do both:

1. Read the diff. Infer what it changes. What code path, what

inputs reach it, what the before/after behavior difference is.

1. Cross-check the stated claim (PR body, commit message) against

your inference. Mismatch is a finding — report it.

1. Verify by running. Drive the app to exercise the changed path,

capture the output, compare to expected.

If there's no stated claim — no PR, no commit message, just a dirty working tree — you still do (1) and (3). Your inference IS the claim. State it explicitly in the report so the author can correct you.

Find the change

This skill verifies a change. If you can't find one, ask.

Establish scope before diffing. A PR or branch may be multiple commits. HEAD~1..HEAD is the tip; if the branch has six commits, you just verified the bookkeeping one and missed the feature. First:

git log --oneline @{u}..HEAD         # or origin/main..HEAD, or $BASE..

If that shows more than one commit, the diff is the full range — git diff @{u}..HEAD, not git diff HEAD~1. State the commit count in your Claim line. A reviewer reading "PASS" should know whether you verified the PR or one commit of it.

Then find the diff:

git diff --stat                      # unstaged
git diff --staged --stat             # staged
git diff @{u}..HEAD --stat           # committed — FULL range, not -1
gh pr diff                           # PR context, if in one

For large diffs, the Bash tool may truncate output — redirect to a file and use Read: git diff @{u}.. > /tmp/diff && Read /tmp/diff. Setting the pager doesn't help; it's tool-side, not git-side.

User might also hand you a branch name, a PR number, a commit range, or a patch file. Use that — and the scope rule still applies: count the commits in whatever they gave you.

No diff, no verification. If all of the above are empty and the user didn't give you a change, say so and stop. Don't verify "the current state of the app" — that's not a change.

Definition of done

You are done when you have evidence — not reasoning — that the changed code does what it should. What counts as evidence depends on what changed:

Most diffs are mixed. Apply the highest applicable bar to each hunk.

Careful with "it's just a config file." If something reads it and does something different, that difference is the surface. A prompt file's surface is the agent that reads it. A CI workflow's surface is the Actions run. A feature flag's surface is the gated feature. Review is the bar only when the sole consumer is human eyeballs.

If your evidence for a runtime change is a script that imports the function and prints its return value — stop. You wrote a unit test. The app never ran. That script proves the function does what the function does, which you already knew from reading it. A reviewer looking at your report sees: you called the code, and the code did what the code does. They could have predicted that from the diff.

(Not the same as sample code against a library's public exports — that IS the DONE for a library change. See What DONE looks like. The tell: does your import go through the package boundary, or reach into src/?)

Process

1. Find the change (above)

2. Read the diff, form a claim

What behavior is different? Not "a function was added" — what does a user or caller see differently? That's the claim you'll verify.

Cross-check against PR body / commit message. If they disagree with the diff, note it now.

3. Get a handle on the app — the discovery ladder

Before investing in the ladder: if the diff touches a callable unit — pure function, utility — call it directly, A/B against parent: same caller on HEAD~1 and HEAD, diff output. No delta where the PR claims one? FAIL, cheap, you saved yourself the ladder. Expected values you derived from reading the diff don't count — that's reading comprehension, run the parent.

Delta present? The mechanism fires. That's not a verdict. The function exists because something calls it and some human sees the result. Go find out what the human sees. That's what the ladder is for — not writing another test, but getting the app running so you can use it.

You will want to stop here. The A/B is clean, the mechanism fires, and running the whole app is work. That's the moment your report becomes a unit test with a narrative attached.

The ladder — for user-facing behavior: UI renders, server responds, CLI prints. Check for existing knowledge first:

`*verifier*` skill exists (.ncode/skills/*verifier*/SKILL.md, legacy .claude/skills/*verifier*/SKILL.md)? → The glob may match multiple verifier skills (e.g. one for CLI, one for GUI). Check each: read its header — what surface does it drive (tmux CLI? HTTP? GUI?)? If that matches the surface your diff reaches, route to it. It knows things you don't — readiness signals, UI gates, env gotchas. If it expects a pre-generated plan, generate one and feed it in. You're done with discovery.

If a verifier's surface doesn't match your diff — a terminal-driving verifier but your diff only touches GUI panels, or an HTTP-probing verifier but your diff is a command-line flag — skip that verifier, not the entire rung. Try the next one. Only skip the rung if no matching verifier exists. A mismatched verifier will FAIL on mechanics unrelated to the change.

If it fails on something that isn't the feature — dev command

changed, build path moved, tool missing — that's the **verifier

being stale**, not the change being broken. Don't FAIL the change

for it. Ask the user (AskUserQuestion) whether to patch the

verifier. If yes: make the minimal edit to its SKILL.md and re-run.

If it's too far gone for a minimal edit, suggest /init-verifiers

to regenerate it.

`run-*` skill exists (.ncode/skills/run-*/SKILL.md, legacy .claude/skills/run-*/SKILL.md)? → It knows how to build and drive the app. Its driver is your handle. Read it, use its launch/interact commands as your primitives. You still plan and judge; it handles the mechanics.

Neither? → Cold start. Survey README, package.json scripts, Makefile, Dockerfile, CI workflows. Find the build command, find the run command, try them.

The run-skill is what makes this reliable. Without one you're

reconstructing "how do I launch this" from scratch every time. For

a CLI or a library that's minutes. For anything with a GUI,

services, or a non-obvious build: you're about to spend most of

your time on mechanics instead of verification.

If the app looks non-trivial, say so before you start

grinding. Tell the user: "No run-skill found — I'll try cold-start,

but /run-skill-generator would make this and every future

verification fast." Then try. If you get through, great. If not,

the user already knows the fix.

Timebox the cold start. You're verifying a change, not writing a run skill. If you're ~15 minutes in without a running, pokeable app: stop, report BLOCKED with exactly where (command, error, what you tried), and hand the user a filled-in prompt:

/run-skill-generator I need to run <app> to verify changes. Got stuck at: <the exact wall you hit>

Don't burn another hour on xvfb for one verification.

If you got through cold start and to a verdict, mention /init-verifiers in your report. You just learned what to check and how — that's a verifier skill. Next time the ladder stops at the top.

4. Plan the minimum interaction

What's the smallest way to make the changed code execute and observe the effect? Not "use the app generally" — target the path:

• Changed a CLI flag? Run with that flag.
• Changed an HTTP handler? curl that route with inputs that hit the branch.
• Changed a UI component? Navigate to where it renders, screenshot.
• Changed error handling? Trigger the error.
• Changed a library function? Something calls it — a CLI command, a

request path, a render. Run that. The caller is where it becomes observable.

Write the plan down before you run. One line per step: what you'll do, what you expect to see.

Now read your plan back. Is every step something CI already ran — typecheck, lint, test files, build, "code review for structural correctness"? Then you haven't planned a verification, you've planned a CI rerun. The green checkmarks on the PR already said those pass. Either find a step that reaches the surface, or stop here: verdict is BLOCKED, report what the surface needs that this environment doesn't have. Don't execute a plan whose only output is "CI still works."

5. Execute and capture

Run each step. Capture output at each step — stdout, screenshots, response bodies. Captured output is evidence. Your memory of what you saw is not.

If your harness touches shared process state — tmux/screen sessions, ports, sockets, lockfiles, global temp — isolate it. tmux -L name, bind :0, mktemp -d. You're running in the same namespace as your host; tmux kill-server takes you with it.

Something unexpected? Don't route around it. Capture it, note it, decide if it's the change or the environment.

6. Report

Inline, in your final message. Shape:

## Verification: <one-line summary of the change>
 
**Verdict:** PASS | FAIL | BLOCKED
 
**Claim:** <what the change is supposed to do — your inference and/or
the stated claim; note any mismatch>
 
**Method:** <how you got a handle — run-skill / verify-skill / cold
start; what you ran>
 
### Steps
 
1. <action> → <observed> — ✅/❌
   <evidence: command + output, or screenshot path>
2. ...