mwguerra/plugins/adversarial-verify

adversarial-verify — Adversarial Self-Verification

Turn "I think I got it right" into "an independent, fault-finding perspective verified it."

Why you need it

Models have a confirmation bias toward what they just produced — they tend to believe they're right. The fraction of single-pass conclusions overturned after an independent adversarial re-check is surprisingly high (often around half in practice). The cheapest insurance is one spawn before delivering, letting a perspective that assumes you're wrong attack it.

When to use

• You just reached an important conclusion ("the root cause is X," "nobody uses this,

it can be deleted," "this needs an N-week refactor").

• You just made a high-risk change (touched a core module, something many things

depend on, or anything before an irreversible operation).

• You just produced something to deliver/ship (a report, a PR to merge, a design

going live).

Simple, reversible, low-risk things don't need verifying every time — that turns into ritual.

How to do it

Step 1 — List what's to be verified as a checklist

Break the conclusion/change into individually checkable claims:

• "The cache at foo.py:42 isn't taking effect"
• "This module has no callers and can be deleted"
• "This change won't affect the login flow"

A vague "I think it's OK overall" can't be verified — break it into concrete claims.

Step 2 — Spawn an independent agent to refute (not review)

The key is the prompt — tell it to refute, not "take a look":

Below is a set of claims to verify. Your task is to overturn them, not affirm them.
Assume each has a 60% chance of being wrong. For each:
- Use ls / grep / read real files / run commands to find evidence that it's wrong
- If a claim rests on a runnable artifact (a query, command, config, code snippet), run the
  LITERAL text as it ships — extract it from the file/doc; never a re-typed "equivalent"
- Only if you can't find a counterexample, grudgingly mark it "tentatively holds"
- Never use guesses or "should be" — it must be real evidence
When unsure, default to "it's wrong."
 
Claims to verify:
1. ...
2. ...

A find-fault prompt catches far more than a review prompt, which tends to go along with you.

Evidence means the literal artifact, run the way the consumer will run it. When a claim depends on a snippet meant to be executed — SQL embedded in a doc, a command in a runbook, a config block, a code example — verify it by running the exact shipped text (extract it from the file; don't re-type a "should-be-equivalent" version). A reviewer who writes the obviously- correct version to check the logic silently repairs the very defect they were sent to find, and reports it sound. (Real case in this suite: a shipped doc query carried an ambiguous column that never parsed, yet passed both an adversarial review and its own test because each ran a corrected paraphrase instead of the literal text — caught only when a later pass executed what actually shipped.) Where a test guards such an artifact, make the test extract and run the artifact, not embed its own copy — a copy diverges; an extraction stays bound to what ships.

Step 3 (if it matters) — Verify from multiple perspectives

For critical conclusions, dispatch different agents with different lenses:

• Correctness: is the logic/fact right?
• Safety: does it introduce risk?
• Reproducibility: does the claimed problem actually reproduce? Run it once.
• Edge cases: extreme inputs, nulls, concurrency — does it break?

Different lenses catch different classes of holes; redundant same-kind agents can't.

Step 4 — Converge

• Overturned claims → drop them, don't stubbornly patch them up.
• Surviving claims → those are deliverable.
• If most got overturned → warning: the original frame may be wrong; rethink rather

than patch claim by claim.

When this pass backs a change you're about to commit and maestro's enforcing gates are installed, record it to the verification ledger so the commit gate is satisfied: bash <plugin>/hooks/lib/ledger.sh mark adversarial pass <ref> "<what survived>" (the gate's denial message supplies the exact <ref> and path). The gate accepts the change only when both a tests-pass and this adversarial-pass record exist for it — or a logged waiver.

Flow

Break into concrete claims
   → spawn an independent agent: "assume you're wrong, refute, must use real evidence"
   → (critical conclusions) one agent per lens
   → drop the overturned, deliver only the survivors

Relationship to other processes

• Stage 2 of deep-work (adversarial review) calls this move to challenge the plan.
• deep-analysis bakes the same refute-by-default asymmetry into its per-flaw skeptics.
• Use it standalone any time you produce an important conclusion before delivering.

The mantra

Assume your own answer has a 50% chance of being wrong, then go prove it wrong. What you can't knock down is what you deliver.