mwguerra/plugins-haiku/regression

regression — did this break anything that already worked?

The verdict is enforced downstream (taskmanager's done-gate blocks done on anything but pass/overridden); this skill produces it. Depth is never sized here: it is read from .maestro/evidence/<diffhash>/sizing.json — run bash ${CLAUDE_PLUGIN_ROOT}/hooks/lib/sizing.sh first if the artifact is missing.

Part 1 — run the suite (current source, never a cache)

1. Discover the project's test command (config, docs/, stack convention); run

it; capture command line + output tail + exit_code= (probe P1 form). No discoverable command → suite.ran=false, the signal is unverified-risk — never a fake pass.

1. Stale-cache guard: clear derived artifacts first (__pycache__, bundler

output, build dirs) or run the cache-bypassing form (python -B) — a stale cache masks a regression as green.

1. A failure is not automatically your diff. Full suite red but the targeted

tests green → bisect by file and check interpreter/resource limits (OOM, time caps, a shared global) before attributing it.

Part 2 — blast-radius probes (selected from the table, never authored)

Probes come from skills/adversarial-verify/references/probe-tables.md; run each selected row literally, attach output. Selection is a lookup on the diff:

Run the selected set prescription.votes times as independent instances (each run executes the probes itself and attaches its own output; no shared transcript). independent_probe: true → the runs are fresh agents, refute-first, given only the diff + the contract files. Any probe with a failing reading = a violated or at-risk contract; cite path:line or the grep output — a criterion without attached output stays failed.

Part 3 — honest coverage + the characterization net

1. List changed ∩ untested (from sizing.json.inputs.untested_touched) as

unverified-risk — confidence is bounded by what was verified.

1. Net predicate (mechanical): untested_touched > 0 AND tier >= T2 → the

net is mandatory for prescription.characterization_net.targets (the hook caps them at the top 3 changed-untested files by max_fan_in). Files beyond the cap stay listed as residual unverified-risk. Tier T1 → the net is off; the risk is still listed.

Capturing a characterization test (per target, fixed steps)

1. Identify the smallest observable seam (function return, CLI stdout+exit,

HTTP status+body, serialized output).

1. Capture against the UNCHANGED code: git stash the working change or read

git show HEAD:<path> — a master captured post-change is a tautology.

1. Inputs: exactly one case per class — happy path · boundary · empty/null ·

error path. Not exhaustive, no fuzzer.

1. Record observed outputs AS the assertions, using the stack's native

snapshot/approval idiom when present, else literal asserts.

1. Nondeterminism: pin at the source (fixed seed / TZ / LC_ALL) or mask

timestamps/UUIDs/ordering. A value you cannot pin is NOT pinned — report it as residual unverified-risk, never flaky-pinned.

1. Prove the test GREEN on the unchanged code (red = it captured nothing —

fail closed), then stage it under the project's characterization/ test folder with the header comment: CHARACTERIZATION — pins current behavior of <path:symbol> as-of <commit>; NOT a correctness claim; do not edit assertions without re-blessing.

After capture, the compare is just part 1: the committed test rides the suite. RED on a later run → classify by the diff's intent: unintended → a genuine regression → fail; intended → re-bless (re-record the assertions in the same commit, reason in verdict_reasoning + a capture.log breadcrumb) — never a logged override against a stale master. Captured paths count as covered and drop out of changed_untested; uncaptured paths of the same symbol stay listed. Every debugged bug becomes a permanent characterization case; cases are only added or re-blessed — removing one is a reviewable git deletion.

Part 4 — the verdict

Emit the JSON in commands/regression.md, write a copy to .maestro/evidence/<diffhash>/regression.json, and insert the regression_checks row when a taskmanager target exists. status='fail' when the suite failed OR any probe reading is violated; else pass. Never emit overridden — that is a logged human action.

What it is NOT

• Not a whole-repo audit (that is deep-analysis); bounded to one change.
• Not the enforcer — taskmanager's done-gate blocks; this emits.
• Not a second verification harness — probes and votes come from

adversarial-verify's tables and the M2 aggregation rules, referenced, never re-implemented.

• Degraded mode (maestro absent): the suite-run + committed characterization

tests still run anywhere; probe selection and capture are maestro-only — report that honestly.