9thlevelsoftware/legion/plan-critique

Input: Plan file(s) for the phase, CONTEXT.md, ROADMAP.md phase details
Output: Ranked list of failure scenarios with root causes and mitigation actions
 
Step 1: Set the scene
  Read ALL plan files for the phase ({NN}-{PP}-PLAN.md).
  Read the CONTEXT.md for phase goal, requirements, and success criteria.
  Read ROADMAP.md for dependencies and the broader milestone context.
  Read `.planning/CODEBASE.md` if it exists (optional):
    - Extract the Risk Areas table
    - Cross-reference each plan's `files_modified` against Risk Areas
    - If overlap with HIGH or MEDIUM risk areas: pre-seed Step 2 failure headlines
      with risk-informed scenarios (e.g., "Phase failed because changes to {risky_file}
      introduced regressions in a high-churn area")
    - If CODEBASE.md absent: skip, no pre-seeding
 
Step 2: Generate failure headlines
  Assume the phase has been executed and FAILED. Generate 3-5 failure headlines
  that describe plausible ways the phase could fail. Each headline should be:
  - Specific to THIS phase (not generic project risks)
  - Grounded in the actual plan content (task instructions, file changes, dependencies)
  - Written as a post-incident headline: "Phase {N} failed because {specific cause}"
 
  Example headlines for a "Plugin Scaffold" phase:
  - "Phase 15 failed because plugin.json schema didn't match Claude Code's validation"
  - "Phase 15 failed because settings.json defaults broke existing user configurations"
  - "Phase 15 failed because the directory structure conflicted with .gitignore patterns"
 
  Do NOT generate vague headlines like "Phase failed due to poor planning" or
  "Phase failed because requirements were unclear."
 
Step 3: Trace each headline to root causes
  For each failure headline, work backward:
  a. Which specific task(s) in the plan would produce this failure?
  b. What assumption in the task instructions is wrong?
  c. What dependency is missing or underspecified?
  d. What edge case does the plan not account for?
 
  Format each trace as:
  | Field | Value |
  |-------|-------|
  | Headline | "Phase {N} failed because..." |
  | Root Cause | {1-2 sentence description} |
  | Plan Section | Task {X} in Plan {NN}-{PP}, "{task_name}" |
  | Requirement | {REQ-ID} or "implicit" |
  | Likelihood | High / Medium / Low |
  | Impact | High / Medium / Low |
  | Mitigation | {specific action to prevent this failure} |
 
Step 4: Rank by risk score
  Score = Likelihood x Impact (High=3, Medium=2, Low=1)
  Sort findings by score descending.
  Top 3 findings are "Critical Risks" — these need action before execution.
  Remaining findings are "Watch Items" — monitor during execution.
 
Step 5: Present pre-mortem findings
  Display to user:
 
  ## Pre-Mortem Analysis — Phase {N}: {phase_name}
 
  **Failure scenarios analyzed**: {count}
  **Critical risks**: {count with score >= 6}
  **Watch items**: {count with score < 6}
 
  ### Critical Risks (action required before execution)
 
  | # | Headline | Plan Section | Risk Score | Mitigation |
  |---|----------|-------------|------------|------------|
  | 1 | {headline} | Plan {PP}, Task {X} | {score} | {mitigation} |
 
  ### Watch Items (monitor during execution)
 
  | # | Headline | Plan Section | Risk Score |
  |---|----------|-------------|------------|
  | 1 | {headline} | Plan {PP}, Task {X} | {score} |

Input: Same as Section 1
Output: Assumption inventory with impact/evidence ratings and challenge actions
 
Step 1: Extract assumptions
  Read through each plan task and identify implicit assumptions. Categories:
 
  a. Technical assumptions
     - "File X exists and has the expected format"
     - "Tool Y supports operation Z"
     - "Path resolution works as described"
     - "No other process modifies these files concurrently"
 
  b. Dependency assumptions
     - "Phase {N-1} output is complete and correct"
     - "Skill X provides the exact API described"
     - "External service Y is available"
 
  c. Scope assumptions
     - "The task description covers all necessary changes"
     - "No other files need updating beyond files_modified"
     - "Edge case X doesn't apply here"
 
  d. Knowledge assumptions
     - "The executing agent understands {concept}"
     - "The user has {prerequisite} in place"
     - "The codebase follows {convention}"
 
  e. Codebase assumptions (if .planning/CODEBASE.md exists)
     - "The detected conventions still apply to the files being modified"
     - "The risk areas identified in CODEBASE.md are current"
     - "The detected stack is compatible with the plan's approach"
     - "No undocumented conventions exist beyond what CODEBASE.md captured"
 
  Aim for 5-10 assumptions. Prioritize assumptions that are:
  - Unstated (not explicitly written in the plan)
  - Load-bearing (if wrong, the plan breaks)
  - Checkable (can be verified before or during execution)
 
Step 2: Rate each assumption
  For each assumption, rate two dimensions:
 
  Impact (if this assumption is wrong):
  - High: Plan fails or produces incorrect output
  - Medium: Plan partially succeeds but needs rework
  - Low: Minor inconvenience, easily corrected
 
  Evidence (that this assumption is correct):
  - Strong: Verified by reading code, confirmed by prior phase output
  - Moderate: Reasonable inference from project context
  - Weak: Not checked, taken on faith, or contradicted by evidence
 
Step 3: Flag critical assumptions
  Critical = High Impact + Weak Evidence
  These are the most dangerous: if wrong, the plan fails, and we haven't checked.
 
  Warning = High Impact + Moderate Evidence OR Medium Impact + Weak Evidence
  These deserve a quick verification before execution.
 
  Accepted = everything else (low risk or well-evidenced)
 
Step 4: Generate challenge actions
  For each Critical and Warning assumption, generate a specific action:
  - "Verify by reading {file} and checking for {pattern}"
  - "Test by running {command} before execution"
  - "Confirm with user: does {condition} hold?"
  - "Add a verify step to Task {X} that checks {condition}"
 
Step 5: Present assumption inventory
  Display to user:
 
  ## Assumption Hunting — Phase {N}: {phase_name}
 
  **Assumptions extracted**: {count}
  **Critical (high impact, weak evidence)**: {count}
  **Warning (needs verification)**: {count}
  **Accepted (low risk or well-evidenced)**: {count}
 
  ### Critical Assumptions (verify before execution)
 
  | # | Assumption | Category | Impact | Evidence | Plan Section | Challenge Action |
  |---|-----------|----------|--------|----------|-------------|-----------------|
  | 1 | {assumption} | {category} | High | Weak | Plan {PP}, Task {X} | {action} |
 
  ### Warning Assumptions (quick check recommended)
 
  | # | Assumption | Category | Impact | Evidence | Challenge Action |
  |---|-----------|----------|--------|----------|-----------------|
  | 1 | {assumption} | {category} | {level} | {level} | {action} |
 
  ### Accepted Assumptions ({count})
  {Collapsed list — one line per assumption with category}

Completeness Check:
 
Input: all plan files for the phase
 
Step 1: Error handling coverage
  For each plan task:
  - Does the task description mention error handling?
  - If the task creates/modifies code: are error paths specified?
  - If the task creates API endpoints: are error response codes listed?
  Flag gaps: "{task} modifies {file} but doesn't specify error handling behavior"
 
Step 2: Edge case enumeration
  For each plan task:
  - Are boundary conditions mentioned (empty inputs, max values, null/undefined)?
  - Are concurrent access scenarios considered (if applicable)?
  - Are timeout/failure scenarios for external dependencies addressed?
  Flag gaps: "{task} interacts with {external service} but no timeout/failure handling specified"
 
Step 3: UI state coverage (if frontend tasks present)
  For each plan task that modifies UI:
  - Are empty states specified?
  - Are loading states specified?
  - Are error states specified?
  - Are success/confirmation states specified?
  Flag gaps: "{task} creates {component} but only specifies the happy-path state"
 
Step 4: API contract coverage (if API tasks present)
  For each plan task that creates/modifies API endpoints:
  - Are all response codes documented (200, 400, 401, 403, 404, 500)?
  - Are error response formats consistent?
  Flag gaps: "{task} creates {endpoint} but only specifies success response"
 
Step 5: Produce completeness findings
  For each gap found:
 
  | # | Gap | Plan | Task | Category | Suggestion |
  |---|-----|------|------|----------|-----------|
  | 1 | {gap description} | {plan PP} | {task X} | error-handling / edge-case / ui-state / api-contract | {specific addition to plan} |
 
  Completeness score: (tasks with adequate coverage / total tasks) × 100
 
  If score < 70: verdict includes CAUTION flag for completeness
  If score < 50: verdict includes REWORK recommendation for completeness
 
Note: Completeness check is advisory — it identifies gaps for the user to decide on.
Not all gaps need fixing (some may be intentionally deferred). The goal is to surface
what might be missed, not to mandate 100% coverage.

Decision Completeness Check:
 
Input: all plan files for the phase
 
Step 1: Validate required harness sections
  For each plan:
  - Must include `<execution_contract>`, `<stop_gates>`, and `<recovery>`.
  - `<execution_contract>` must mention:
    read-before-write, evidence-before-action, minimal diff, verify-before-report,
    role, task, scope, allowed tools/actions, forbidden actions, verification
    criteria, and final result format.
  - `<stop_gates>` must use the literal status `BLOCKED`.
  - `<recovery>` must mention `git diff`, modified files, verification state,
    and the plan result artifact.
 
Step 2: Find implementer-owned decisions
  Flag a decision-completeness gap when a task asks the executor to:
  - choose architecture or design pattern
  - infer paths, APIs, schemas, types, helpers, or imports
  - decide validation behavior, error handling, compatibility behavior, or UI states
  - invent test files, test commands, mocks, fixtures, or acceptance checks
  - use phrases such as "implement as appropriate", "use existing helpers"
    without named files/symbols/patterns, "add tests" without paths and
    commands, or "verify manually"
  - rely on "confidence", "looks good", or unscripted review instead of proof
 
Step 3: Rate each gap
  - High impact: executor could choose a different architecture/API/path/test
    shape than the planner intended, or success cannot be proven.
  - Medium impact: executor still has a constrained path, but one edge/error
    case or helper reference needs clarification.
  - Low impact: wording polish; no implementation choice changes.