jpicklyk/task-orchestrator/session-retrospective

Session Retrospective

Structured post-implementation analysis. Evaluates the run across five dimensions, persists findings in MCP, and keeps cross-session trend memory to surface improvement proposals.

Calls are 4.0 tool calls; shapes live in `../_generated/call-shapes.md`. Every answer is the response envelope (ok, data or error). A write is {ops: [...]} of flat elements (one per to-call block here). Results come back as data.results[i] in request order with data.summary {ok, failed}; read ops of an ops tool (config get, validate, effective) answer there too. These calls are non-atomic: if any element fails, even in a one-element call, the answer is ok: false with top-level error.code: partial_failure (no detail, no fix) beside the full data, and the cause is that element's data.results[i].error (code, message, detail, fix). Any other top-level error.code is a call-level rejection (envelope, unknown field, access).

How the trigger works

A retrospective is owed per closed run that is not marked retro. Two hooks watch for it:

• retro-trigger (PostToolUse on runs_write) acts only on a successful op: close.
• retro-backstop (Stop) re-checks at the end of every turn, so a run closed elsewhere (another session, a subagent) is still found. A closed run is a confirmed boundary, so the backstop may dispatch as well as nudge.

Both ask REST runs_read list with unmarked: retro and endedSince = now - 24h, and read each run's terminal count from runs_read frontier (data.counts.terminal), for at most 5 runs per pass. In mode dispatch with a terminal count at or above dispatchThreshold (default 3) they call runs_write mark {runId, key: retro} (REST only; first write wins, which is the cross-session dedupe; marked: false means another session won, so they stay silent) and emit the dispatch directive. Below the threshold, in nudge mode (the default), or when the mark fails, they nudge once per run per session and never mark. There is no cross-session marker file (the server mark is the dedupe); per-session state (recorded closes, runs already nudged) lives in os.tmpdir()/task-orchestrator/<session>/retro.json. A run whose frontier read fails is skipped that pass, and a close made by a subagent is recorded for the main session's backstop, not acted on. Without an API URL or a project rootId the backstop only nudges once per session, from the closes the trigger recorded.

retro-ack.mjs is the manual acknowledgement: node retro-ack.mjs --run <id> [--run <id>]... sends the same runs_write mark {runId, key: retro} over REST for each run (Step 8b calls it). It prints one line per id (marked <id>, already marked <id>, mark failed <id>: <code>, or not marked <id>: no REST API configured) and always exits 0.

Director sessions. A director never runs a retrospective in its own context, so for it both hooks always dispatch (mode dispatch, threshold 0, whatever the config says; off stays off): every closed unmarked run it sees is marked first and then dispatched, so a later session does not dispatch it again. Without REST, director-watch dispatches once per run per session and only for a run closed within the last 24 h. The dispatch is ONE background agent that invokes this skill with a repeated --run.

The settings (mode: nudge | dispatch | off, and dispatchThreshold) are read ONLY from extensions.claude-code.retrospective; a top-level retrospective: block is ignored and cooldownMinutes is gone. Headless iterations, to_mod_retro and mode: off make both hooks inert.

Honest limit. The mark is REST-only. Without a resolvable REST API URL (Step 8b prints not marked ...: no REST API configured), a manually run retrospective leaves its run unmarked, so it can nudge or dispatch again in a later session inside the 24 h window; Step 5b's run:<runId> search keeps that repeat from creating a second retrospective item.

Step 0 - Mode Check

• --dry-run sets DRY_RUN = true: perform steps 1-4 and render the report (step 9); skip steps 5-8 (no item creation, no memory updates). Announce: **Dry run** - no items created, no memory updated.
• --run <runId> scopes the analysis to a run (step 1a). It repeats (--run A --run B): the scope is the union of the runs' members, and one retrospective covers them all. It wins over a root UUID.
• --deep is handled in 4.3.

Step 1 - Gather Scope

Collect the distributed session-tracking notes of the items in scope.

1a. Identify items in scope

`--run <runId>` (once per run when repeated; union the members). The run view gives rootId and scopeId:

{"tool": "runs_read", "select": "get", "args": {"runId": "<runId>"}}

Members are the distinct itemIds of the run's seat assignments, ended ones included (coordinator rows carry no itemId), plus the ids in the frontier's waiting and inFlight lists. A closed run's frontier lists no terminal members, so the seats read is the source for finished work. Seat itemIds are full UUIDs and frontier ids are 8-hex short ids: dedupe on the first 8 hex characters.

{"tool": "runs_read", "select": "seats", "args": {"runId": "<runId>", "ended": true}}

{"tool": "runs_read", "select": "frontier", "args": {"runId": "<runId>"}}

When that member set is empty and the run has a scopeId, fall back to the scope root:

{"tool": "items_read", "select": "overview", "args": {"itemId": "<scopeId>"}}

A root UUID is the authoritative scope (also the dispatched background-agent case). List its subtree; run no fallback scan:

{"tool": "items_read", "select": "list", "args": {"ancestorId": "<root-uuid>", "limit": 200}}

Neither argument. Check for a project root: the ## Project Scope section of the SessionStart context, or project.rootId in the project's .taskorchestrator/config.yaml. A personal root is not a project root and takes the unscoped branch. With a rootId, scope both fallback calls to that subtree so other projects sharing the DB are not conflated; without one, omit ancestorId.

{"tool": "items_read", "select": "context", "args": {"ancestorId": "<rootId>"}}

{"tool": "items_read", "select": "list", "args": {"role": "terminal", "sort": "modifiedAt", "order": "desc", "limit": 20, "modifiedAfter": "<ISO now minus 24h>", "ancestorId": "<rootId>"}}

The 24-hour window is the modifiedAfter filter: list results carry no timestamps unless asked for, so staleness cannot be judged afterwards.

1b. Collect distributed notes

For each item in scope (up to 20), list its notes by key. A list carries no bodies, so get each note that exists.

{"tool": "notes_read", "select": "list", "args": {"itemId": "<uuid>", "keys": ["session-tracking", "delegation-metadata"]}}

{"tool": "notes_read", "select": "get", "args": {"itemId": "<uuid>", "key": "session-tracking"}}

session-tracking holds per-item outcome, files changed, deviations, friction, observations and test results; delegation-metadata (optional) holds the orchestrator-recorded model and isolation (structured provenance line or legacy prose).

1c. Early exit

If no items are in scope, or no item has a session-tracking note, exit with: No implementation run data found. Nothing to retrospect - run /implement first, then try again.

Step 2 - Aggregate Note Data

Across the collected session-tracking notes: total item count and outcome distribution (success, partial, failed, skipped); combined deduplicated files list; combined friction list; combined observations; test result totals. Run the health read of 1a (context) for the current state snapshot when not already done.

From delegation-metadata notes extract model per delegation (haiku, sonnet, opus) and isolation (inline, worktree); they feed 3b.

Note volume (only with the director tools). Skip silently without them. Make two director-profile reads over one window: since is the createdAt of the newest session-retrospective item for this project, else 7 days ago, and the window is capped at 31 days.

{"tool": "report", "select": "notes", "args": {"rootId": "<rootId>", "groupBy": "role", "since": "<window start>"}, "profile": "director"}

{"tool": "report", "select": "notes", "args": {"rootId": "<rootId>", "groupBy": "noteKey", "since": "<window start>"}, "profile": "director"}

Compute three figures: the review role's share of total bytes, the review-checklist avgBytes, and the largest maxBytes of any note key. Compare them with the 2026-11-02 baseline (review share 31%, review-checklist average 6.5 KB, largest note 37 KB) and record the three figures in the session-metrics note (5c). Basis caveat: report notes counts every note.upserted write (a rewrite counts again, avgBytes is per write), while the baseline measured final note bodies once, so a run with many rewrites reads higher; say so beside the figures when rewrites is not small.

Structured form. A note whose first line matches ^adapter=\S+( [a-z-]+=\S+)+$ is a structured provenance line, not legacy prose: split on spaces, then each token on key=value. Each seat:model pair inside seats= is one delegation, typed by seat rather than guessed from prose - planner: architecture/opus; implementer, test-author: sonnet; declarations-extractor: code reading/sonnet; reviewer: opus; any other seat is recorded but not scored. Isolation comes from isolation=. model= is ignored whenever seats= is present. Field order is fixed: adapter= run= seats= model= isolation= agents= tokens= duration= deferred= in-run-edges= orchestrator-turns=. Example:

adapter=claude-workflow run=r-20260928-b2d seats=planner:opus,implementer:sonnet,declarations-extractor:sonnet,test-author:sonnet,reviewer:opus model=sonnet isolation=worktree:.claude/worktrees/feat-b2-front-door agents=5 tokens=412800 duration=1860000 deferred=0 in-run-edges=1 orchestrator-turns=3

tokens= / duration= may read see:<short>: the run total lives on that item, so count it once and never sum it per item (in the per-run table show the run's first-item value, never a see: reference). An optional second line extra-seats=<seat>:<model>:<tokens>,... lists seats dispatched outside the run; score each like a seats= pair, typing the seat after stripping a trailing fix-cycle suffix -a<n> (reviewer-a1 scores as reviewer). Legacy form: free-form prose naming model and isolation, parsed as before; a run mixing both forms scores both.

Step 3 - Evaluate Across Dimensions

3a. Schema Effectiveness

For each item, examine its notes (1b): which schema-required notes exist with non-empty content; size per note (<50 tokens sparse, 50-500 appropriate, >500 potentially verbose for status-type notes, specification notes exempt); items missing required notes. Score: fraction of expected notes that exist with appropriately sized content.

3b. Delegation Alignment

If delegation-metadata notes exist, cross-reference the expected model: haiku for MCP bulk ops, materialization and simple queries; sonnet for code reading, implementation and test writing; opus for architecture, complex tradeoffs and multi-file synthesis.

• Flag misalignments (opus for bulk ops, haiku for architecture), scored per seat delegation for a structured note (one score per seat:model pair).
• A substituted= field is an allowlist substitution, not a misalignment. A model-source=self-report field is noted separately and not folded into the misalignment count.
• Score: fraction of delegations matching the expected model. With no delegation-metadata, note "delegation metadata not recorded" and skip scoring.