tmdgusya/engineering-discipline/long-run

Long Run Harness

Orchestrates multi-day execution of complex tasks through a milestone pipeline. Each milestone passes through plan-crafting → run-plan → review-work with checkpoints between milestones for recovery from interruptions.

Core Principle

Long-running execution must be resumable, auditable, and fail-safe. Every state transition is persisted to disk before the next action begins. If execution stops for any reason — rate limit, crash, user pause, context loss — it can resume from the last checkpoint without repeating completed work.

Hard Gates

1. Milestones must exist before execution. Either from milestone-planning skill or user-provided. Never generate milestones inline during execution.
2. State file must be updated before and after every milestone. No in-memory-only state. If it's not on disk, it didn't happen.
3. Each milestone must complete the full pipeline. plan-crafting → run-plan → review-work. No shortcuts. No skipping review-work "because it looked fine."
4. Failed milestones block dependents. If M2 depends on M1 and M1 fails review, M2 does not start. Period.
5. Autonomous by default — no approval gates between milestones. The user approved the milestone plan once, at milestone-planning's lock. From then on, milestones proceed plan → run → review → checkpoint without pausing for confirmation. Stop and ask ONLY when: a milestone exhausts its retry budget (failed), Mid-Execution Correction requires a scope decision, integration escalation triggers, or loaded state is inconsistent. Post a brief status report at each phase transition so the user can interrupt anytime — state files make every pause resumable.
6. Never modify completed milestones. Once a milestone passes review-work, its files are locked. If a later milestone needs changes to earlier work, that is a new milestone.
7. Checkpoint after every milestone completion. Write a checkpoint file recording what was done, test results, and review verdict before proceeding.

When To Use

• After milestone-planning has produced a milestone DAG
• When the user says "long run", "start long run", "execute milestones", or "run all milestones"
• When resuming a previously paused long run session

When NOT To Use

• When milestones don't exist yet (use milestone-planning first)
• When there's only one milestone (use plan-crafting + run-plan directly)
• For quick tasks that don't warrant multi-phase execution

Input

1. Harness state directory path — e.g., docs/engineering-discipline/harness/<session-slug>/
2. The directory must contain state.md and milestones/*.md files

If no state directory exists, ask the user if they want to run milestone-planning first.

Process

Phase 1: Load and Validate State

1. Read state.md from the harness directory
2. Read all milestone files from milestones/
3. Validate:

• All milestones referenced in state.md have corresponding files
• Dependency DAG is valid (no cycles, topological sort possible)
• No milestone is in an invalid state (e.g., "executing" without a plan file)

1. Determine current position:

• Which milestones are completed?
• Which milestones are ready to start (all dependencies met)?
• Is this a fresh start or a resume?

1. Present status to the user:

## Long Run Status: [Session Name]
 
**Progress:** N/M milestones completed
**Current phase:** [planning M3 | executing M3 | reviewing M3 | ready to start M3]
**Next up:** [M3, M4 (parallel)]
 
Completed: M1 ✓, M2 ✓
In progress: M3 (executing)
Pending: M4, M5

1. If the loaded state is consistent, proceed immediately — the status display is a report, not a gate. Ask the user only if validation in step 3 found inconsistencies or a failed milestone blocks the path forward.

Phase 2: Milestone Execution Loop

For each milestone in topological order:

┌─────────────────────────────────────┐
│         Milestone Pipeline          │
│                                     │
│  ┌──────────┐    ┌─────────┐        │
│  │  Plan    │───→│  Run    │        │
│  │ Crafting │    │  Plan   │        │
│  └──────────┘    └────┬────┘        │
│                       │             │
│                  ┌────▼────┐        │
│                  │ Review  │        │
│                  │  Work   │        │
│                  └────┬────┘        │
│                       │             │
│              ┌────────▼────────┐    │
│              │   PASS?         │    │
│              │  Yes → checkpoint│    │
│              │  No  → retry    │    │
│              └─────────────────┘    │
└─────────────────────────────────────┘

Step 2-1: Gate Check

Before starting a milestone:

1. Verify all dependency milestones have status completed
2. Verify no file conflicts with in-progress parallel milestones
3. Update state.md: set milestone status to planning
4. Update execution log with timestamp

Step 2-2: Plan Crafting Phase

1. Compose a Context Brief from the milestone definition:

• Goal → from milestone file
• Scope → files affected from milestone file
• Success Criteria → from milestone file
• Constraints → inherited from the parent problem + completed milestone context
• Completed milestone context contract: From each completed predecessor, include ONLY:
• Files created/modified (from checkpoint's "Files Changed" list)
• Interface contracts established (function signatures, API shapes, type definitions)
• Success criteria that were verified as met
• Do NOT include: execution logs, review documents, worker/validator output, or full checkpoint contents
• Note: Context Briefs composed from milestone definitions omit the Complexity Assessment section, since routing has already been determined by the milestone-planning phase. The brief goes directly to plan-crafting without re-routing.

1. Invoke the plan-crafting skill pattern:

• Create a plan document at docs/engineering-discipline/plans/YYYY-MM-DD-<milestone-name>.md
• The plan must satisfy all milestone success criteria
• The plan must not modify files outside the milestone's scope

1. Update state.md: record plan file path for this milestone
2. Report, don't gate: Post a brief plan summary (goal, task count, files touched) as a progress report and proceed directly to execution. Do not wait for approval.

Step 2-3: Run Plan Phase

1. Update state.md: set milestone status to executing, increment Attempts counter by 1
2. Execute the plan using the run-plan skill pattern:

• Worker-validator loop for each task
• Parallel execution for independent tasks
• Information-isolated validators

1. If run-plan reports failure after 3 retries on any task:

• Update state.md: set milestone status to failed
• Record failure details in execution log
• Stop and report to user. Do not proceed to dependent milestones.

1. If all tasks complete: proceed to review phase

Step 2-4: Review Work Phase

1. Update state.md: set milestone status to validating
2. Invoke the review-work skill pattern:

• Information-isolated review against the plan document
• Binary PASS/FAIL verdict

1. If PASS:

• Update state.md: set milestone status to completed
• Write checkpoint file (see Checkpoint Format below)
• Update execution log
• Proceed to next milestone

1. If FAIL:

• Record review findings in execution log
• Retry decision (based on `Attempts` counter in state.md, which persists across crashes):
• If Attempts == 1: return to Step 2-3 with review feedback (re-execute same plan)
• If Attempts == 2: return to Step 2-2 (re-plan with review feedback as constraint)
• If Attempts >= 3: set status to failed, stop, report to user

Step 2-5: Cross-Milestone Integration Check

After a milestone passes review-work but before writing the checkpoint, verify that the milestone's output integrates correctly with all previously completed milestones:

1. Run the project's highest-level verification (from state.md's Verification Strategy or rediscover using plan-crafting's Verification Discovery order)
2. Check cross-milestone interfaces: If the completed milestone defines or consumes interfaces from predecessor milestones, verify they are compatible (function signatures match, API contracts hold, types align)

If integration check passes: Proceed to checkpoint.

If integration check fails — Cross-Milestone Failure Response:

The milestone passed its own review-work (internal correctness) but breaks integration with other milestones. This is a boundary problem.

1. Diagnose (attempt 1):

• Read the failure output
• Identify which interface boundary or interaction is broken
• Determine if the fix belongs to the current milestone or requires a corrective milestone
• If fixable within current milestone scope: dispatch a targeted fix worker → re-run review-work → re-run integration check
• If the fix is outside current milestone scope: proceed to escalation

1. Diagnose (attempt 2):

• If the first fix didn't resolve it, re-analyze
• Apply a second targeted fix
• Re-run integration check

1. Escalate to user (after 2 failed attempts):

• Report: which milestones are involved, what integration boundary failed, what fixes were tried
• Options: add corrective milestone, rollback to checkpoint, accept and continue (user acknowledges the integration gap)
• Log the user's decision in state.md execution log

Step 2-6: Checkpoint

After a milestone passes review:

Write checkpoints/M<N>-checkpoint.md:

# Checkpoint: M<N> — [Milestone Name]
 
**Completed:** YYYY-MM-DD HH:MM
**Duration:** [time from planning start to review pass]
**Attempts:** [number of plan-execute-review cycles]
 
## Plan File
`docs/engineering-discipline/plans/YYYY-MM-DD-<name>.md`
 
## Review File
`docs/engineering-discipline/reviews/YYYY-MM-DD-<name>-review.md`
 
## Test Results
[Full test suite status at checkpoint time]
 
## Files Changed
[List of files created/modified in this milestone]
 
## State After Milestone
[Brief description of system state — what works now that didn't before]

Phase 3: Parallel Milestone Execution

When multiple milestones have all dependencies satisfied and no file conflicts:

1. Identify parallelizable milestone group
2. Run plan-crafting for ALL parallel milestones first (sequentially — plans are lightweight)
3. Post all plan summaries as a single progress report (no approval wait), then dispatch each milestone's pipeline concurrently:

• Each milestone runs run-plan → review-work
• Each runs in a worktree (isolation: "worktree") to prevent file conflicts
• After both complete and pass review, merge worktrees back

1. If either fails: handle independently (the other can continue if no dependency)

Worktree merge protocol:

1. Both milestones pass review in their respective worktrees
2. Check for file conflicts between worktree changes
3. If no conflicts: merge sequentially (M_lower first, then M_higher)
4. If conflicts detected: stop, report to user, request manual resolution
5. After merge: run full test suite on merged result
6. If tests fail: stop, report to user

Phase 4: Completion

After all milestones are completed (including the Integration Verification Milestone from milestone-planning):