xiduzo/wtf/wtf-verify-trace

Verify Trace

Start an existing Trace as a QA engineer.

The verification contract is the Trace's Scenario Claim. Verify exactly the claimed scenarios — no more, no less. The scenario text is canonical in the parent Feature body. Fetch it from there. Edge cases outside the claim belong to other Traces in the Trace Plan. Do not probe them here.

Two verification methods exist. When the repo has a Gherkin runner, project the claimed scenarios into a temporary .feature file and execute them. When no runner exists, or steps are unbound, walk each scenario against the running software. The verdict states which method verified each scenario.

Read references/qa-verdict-guide.md before you start. It defines the status symbols, the method values, the verdict options, and the Test Mapping table format used throughout this skill.

Process

0. GitHub CLI setup

Run steps 1–2 of ../references/gh-setup.md (install check and auth check). If gh is not installed or not authenticated, stop. Extensions are not required for this skill.

If this skill was invoked from wtf.implement-trace or another skill that already ran gh-setup this session, skip this step.

1. Identify the verification scope

Call AskUserQuestion (per ../references/questioning-style.md):

• question: "Are you verifying a single Trace or a full Feature?"
• header: "Scope"
• options:
• Single Trace → verify one Trace's Scenario Claim
• Full Feature → verify all child Traces of a Feature using its sub-issues

If Single Trace:

Call AskUserQuestion (per ../references/questioning-style.md):

• question: "Which Trace are you testing?"
• header: "Trace"
• options: from recent open Trace issues (list per ../references/issue-classification.md). Prefer issues labeled implemented.

Walk Trace → Feature per ../references/spec-hierarchy.md. From the Trace, extract the Scenario Claim, the synced scenario copy, the Spine position, Contracts, Observability, and DoD. From the Feature, extract the claimed story with its canonical scenarios, plus the Trace Plan.

Apply the absent-label gate from ../references/lifecycle-labels.md for the implemented label on the Trace.

Recommended skill: wtf.implement-trace. Header: Implement first?.

On Implement first → follow wtf.implement-trace and pass the Trace number as context. On Verify anyway → continue.

If Full Feature:

Call AskUserQuestion (per ../references/questioning-style.md):

• question: "Which Feature are you verifying?"
• header: "Feature"
• options: from open feature issues

Fetch all sub-issues of the Feature using gh sub-issue list <feature_number> per the cookbook in ../references/gh-setup.md. This returns the authoritative list of children. Do not search by label or title matching. Split the children by kind per ../references/issue-classification.md: Trace children, and legacy Task children.

Verify Traces sequentially. Traces of one Feature build on each other's Spine, and sibling Traces may have landed in either order. Verify in Trace Plan order regardless — it is the order the Spine was designed in. If a Trace does not appear in the plan, fall back to issue-number order. Do not parallelize Traces of one Feature: they share a checkout, and a later Trace's scenarios can depend on an earlier Trace's code.

Spawn one sub-agent per Trace, one at a time. Apply the rules in ../references/subagent-protocol.md for every Agent call. Specifically for Full Feature mode:

• Read skills/wtf.verify-trace/SKILL.md at spawn time and paste steps 2–8 into each sub-agent prompt (inline instructions — sub-agents cannot load this skill by name).
• Override interactive prompts as follows:
• Claim confirmation (step 3) → skip, proceed with the full Scenario Claim
• Drift gate (step 3) → do not resolve alone. Return as NEEDS_INPUT to the orchestrator
• Per-scenario pass/fail in the interpretive walk (step 6) → return as NEEDS_INPUT to the orchestrator
• Bug-filing prompts → defer to the aggregated step 10
• Mandatory label (non-skippable): gh issue edit <trace_number> --add-label "verified" after verification passes. The sub-agent also ticks the Trace's entry in the Feature's Trace Plan (step 8, Tick the Trace Plan) — non-skippable for the same reason.

Resolve any NEEDS_INPUT responses for a Trace before you start the next Trace.

Legacy Task children are legacy Traces (see ../references/issue-classification.md). Their Gherkin lives in the Task body — extract it per the legacy sections in ../references/spec-hierarchy.md. Verify legacy Tasks interpretively (step 6 flow, no drift gate — the Task body is its own scenario source). Schedule them with the file-conflict coloring in ../references/conflict-graph.md, after the Trace sequence completes.

After all children finish, aggregate into a feature-level summary. Include child counts, pass/fail/blocked totals, the partition status of each story, and whether the Feature is complete. Present it.

2. Load the QA steering document

Load docs/steering/QA.md per the strict consumer-side load in ../references/steering-doc-process.md (recommended skill: wtf.steer-qa). Apply its test strategy, coverage thresholds, definition of done, and known flaky areas silently throughout this session.

Also record any Gherkin runner or BDD tooling the document declares. Step 4 uses that declaration first.

3. Establish the verification contract

From the Trace body, extract:

• The claimed scenario names (the Scenario Claim)
• The synced courtesy copy in the collapsed <details> block
• Contracts and Observability (step 7 uses these)

From the Feature body, extract the canonical text of each claimed scenario, under the Trace's story.

If a claimed scenario name does not exist in the Feature body, stop. The claim is broken. Report the missing name and recommend /wtf.refine on the Feature to repair the Trace Plan.

Drift gate. Compare the synced copy against the canonical Feature text, with whitespace normalized. If any claimed scenario differs, stop before you verify. The sync is stale. The Feature wins. Present the differences, then call AskUserQuestion (per ../references/questioning-style.md):

• question: "The Trace's synced scenario copy differs from Feature #[N]. The Feature body is canonical. How do you want to proceed?"
• header: "Stale sync"
• options:
• Verify against the Feature text → continue with the canonical text. Note the stale sync in the verdict and recommend /wtf.refine to resync the copy
• Halt → exit. Run /wtf.refine on the Feature first, then verify again

Confirm the claim. Call AskUserQuestion (per ../references/questioning-style.md):

• question: "The Scenario Claim names [n] scenarios. I will verify exactly these — no more, no less. Correct?" (replace [n] with the actual count)
• header: "Claim"
• options:
• Yes — verify the claim → proceed to testing
• The claim is wrong → stop. Scenario ownership changes go through /wtf.refine on the Feature, not through an ad-hoc edit here

4. Detect a Gherkin runner

Decide the verification method before you test. Precedence:

1. A runner declared in docs/steering/QA.md (step 2) wins.
2. Otherwise probe the dependency manifests:

``bash grep -nE '"(playwright-bdd|@cucumber/cucumber|cucumber-js)"' package.json 2>/dev/null grep -nE '(behave|pytest-bdd)' pyproject.toml setup.cfg Pipfile requirements*.txt 2>/dev/null ``

1. Confirm that step definitions exist:

``bash ls features/step_definitions 2>/dev/null # cucumber-js default ls features/steps 2>/dev/null # behave default git ls-files | grep -iE '(^|/)(steps?|step_definitions)(/|\.)' | head -20 ``

Selection rules:

• If QA.md names a runner, use that runner.
• If exactly one manifest probe hits, use that runner.
• If several runners hit and QA.md is silent, ask the user which one to use (headless: NEEDS_INPUT).
• If no runner hits, or the runner has no step definitions at all, use the interpretive fallback for every claimed scenario. Skip step 5.

Note: this probe covers the common runners. A repo can execute Gherkin in another way — docs/steering/QA.md is the place to declare that.

5. Project and execute (runner found)

Build the ephemeral projection:

1. Create a scratch directory outside the repo: TMP=$(mktemp -d).
2. Write $TMP/trace-<trace_number>.feature with the Write tool. Content: a comment header, one Feature: line, then the canonical text of each claimed scenario, verbatim from the Feature body:

```gherkin # Ephemeral projection from Feature #<feature_number> for Trace #<trace_number>. Do not commit. Feature: <feature title> — Scenario Claim of Trace #<trace_number>

Scenario: <claimed scenario, canonical text> ```

Keep the projection outside the repo and never commit it. The Feature body stays the only canonical scenario source.

1. Bind check (dry run), then run. Bind the projection to the project's existing step definitions:

1. If the dry run reports undefined steps, record which scenarios contain them. Move those scenarios to step 6 (interpreted). Do not write new step definitions during verification. A missing binding is a finding for the verdict, not QA work.
2. Run the bound scenarios. Capture per-scenario pass/fail and the failure output.
3. Record each executed result per the recording rules in step 6, with method executed. On ❌, use the runner output as the failure details, then offer a bug report per the step 6 flow.
4. Remove the scratch directory after the run.

Executed results are the primary verdict evidence. Do not re-walk a scenario the runner already executed.

6. Walk the remaining scenarios (interpreted)

Scenario set: every claimed scenario when no runner was found, plus the scenarios with unbound steps from step 5.

For each scenario, one at a time, call AskUserQuestion (per ../references/questioning-style.md):

1. Present it as a concrete test case. Restate the Given/When/Then in plain language. Walk it against the running software.
2. Ask "Did this scenario pass?" — header Result:

• Yes ✅ → mark ✅ in the running Test Mapping table. Set bug filed to —
• No ❌ → record failure (see below)
• Blocked 🚫 → record blocker (see below)
• N/A or Conditional ⚠️ → record condition (see below)

On No ❌: call AskUserQuestion (per ../references/questioning-style.md):

• question: "What actually happened?"
• header: "Failure details"
• options: from plausible failure modes inferred from the scenario (e.g. "No error shown", "Wrong data returned")

Record findings with repro steps. Then call AskUserQuestion (per ../references/questioning-style.md):

• question: "Would you like to file a bug report now?"
• header: "File bug?"
• options:
• File now → run wtf.report-bug immediately with the Trace number and the failing scenario name (default). It links the failing scenario to the Trace. Mark bug filed as yes
• Continue and file later → defer. Mark bug filed as no