kevinpatrickrobbins/codebase-audit/tickets-audit

Tickets Audit

A backlog of tickets rots in two ways. Syntactically — format drifts from the standard, labels lie about status, dependencies point to closed or non-existent issues. And semantically — the code moves on, features get silently shipped or removed, and the ticket text no longer matches reality.

This skill audits both. The output is a markdown report under docs/audits/ that a human can act on: which tickets to close, which to update, which to refile, which to investigate.

The flagship dimension is codebase relevance — for each ticket, an agent reads the ticket plus the actual code and judges whether the work is still real. That's where most of the value comes from. The other dimensions are syntactic gates run more cheaply.

When to use

This skill runs only when the user explicitly invokes `/tickets-audit`. Do not auto-trigger on ticket-related conversations.

Once invoked, the user wants one of:

• A full backlog audit ("audit our tickets")
• A focused stale-check ("are these still relevant?")
• A silent-completion sweep ("have any of these shipped without being closed?")
• A grooming pass before planning or a release

If the user invoked this skill but their actual intent is creating a new ticket, migrating TICKETS.md to a different backend, or auditing architecture contracts, redirect them to [[tickets-and-logbook]], [[tickets-migration]], or [[contracts-audit]] respectively.

Source detection

Detection is not just file-presence — read project docs first. Many projects have a TICKETS.md that is a stub pointer to a real backend (Airtable, Linear, etc.), or a CLAUDE.md / AGENTS.md that explicitly forbids one source and routes to another. File existence alone is a misleading signal.

Step 1 — read project intent. Open CLAUDE.md, AGENTS.md, README.md, and docs/TICKETS.md (read the whole file if it exists — likely short if it's a stub). Look for explicit statements like "tickets live in X", "no GitHub Issues", "migrated to Airtable", or backend ID patterns: Airtable base app[A-Za-z0-9]{14} / table tbl[A-Za-z0-9]{14}, Linear team prefixes, etc. The project's own docs are the highest-confidence signal — they override file/remote heuristics.

Step 2 — fall back to file / remote detection:

1. Airtable — if Step 1 yielded an Airtable base ID, or the repo has .airtablerc / AIRTABLE_BASE_ID env reference, use Airtable. The docs/TICKETS.md body often contains the base + table IDs explicitly — grep there before asking the user. See references/airtable.md.
2. GitHub Issues — if gh repo view succeeds and gh issue list --state all --limit 1 returns at least one issue, AND the project docs don't forbid GitHub Issues, use GitHub Issues. See references/github-issues.md.
3. TICKETS.md (flat-file) — only when docs/TICKETS.md exists AND is NOT a stub. Heuristics: >100 lines AND contains actual ## TICKET-NNNN (or similar) heading-style entries. In this case — and only this case — recommend at the top of the report that the project migrate via [[tickets-migration]]. Never recommend migration when TICKETS.md is a stub pointing at a real backend — that migration has already happened. See references/tickets-md.md.

If more than one real source exists (e.g., GitHub Issues and a real TICKETS.md), audit both and produce a single combined report. Note the duplication as a finding.

Per-source mechanics live in the references:

• GitHub Issues: read references/github-issues.md
• Airtable: read references/airtable.md
• TICKETS.md: read references/tickets-md.md

The six dimensions

Run in this order — cheapest and most objective first, semantic codebase reasoning last, cross-ticket duplication last of all (it depends on dimension 5's verdicts). This lets you batch the syntactic checks across all tickets quickly, then concentrate the expensive judgment work on the tickets that are worth it.

1. Format compliance

The project's own schema takes precedence over the canonical one. Before applying any checklist, look for project-specific schema documentation in docs/TICKETS.md, AGENTS.md, CLAUDE.md, or a project-level skill. If the project documents its own ticket schema — its required fields, its status taxonomy, its label conventions — audit against THAT. The canonical [[tickets-and-logbook]] schema below is the fallback only when no project schema is documented.

When a project names a concept differently (e.g. Scope instead of Problem, or Acceptance instead of Acceptance Criteria), map the canonical concept to the project's field and check whether the concept is covered, not whether the field is named the canonical way. A "missing Problem" finding when the project's Scope field carries the problem is noise — it floods the report and trains the human to ignore it. Only flag truly-missing concepts.

Similarly, the canonical status names (proposed, ready, in-progress, blocked) are illustrative. Remap to the project's actual Status single-select values when judging staleness or readiness. The verdict structure of dimensions 3 and 5 still applies — only the labels change.

Canonical fallback (use only when the project has no documented schema):

• Required sections: Problem, Approach, Dependencies, Acceptance Criteria, Testing, Logbook
• Exactly one thematic label and one status label
• Title style: short imperative, no ticket ID prefix

Catches: truly-missing concepts (not naming differences), duplicated sections, malformed acceptance criteria (not checkbox lists in projects that require checkboxes), label-count violations against the project's documented convention, status values that don't match the project's documented taxonomy.

2. Dependency integrity

Builds the dependency graph from Depends on: #N lines.

Catches: refs to issues that don't exist; refs to closed issues from open ones (which may now be unblocked); cycles; ready-labeled issues with unresolved dependencies; blocked-labeled issues whose blockers are closed.

3. Status & logbook drift

Cross-references status labels with actual activity.

Catches: in-progress tickets with no commit, comment, or logbook entry in the last 30 days; closed tickets with no logbook entry; logbook entries that reference issues that don't exist; commits referencing #N for issues still labeled proposed.

4. Content quality & gaps

Reads the ticket body for substance.

Catches: vague acceptance criteria ("works well", "is fast"); missing Testing section content; Approach section that's just one bullet for non-trivial work; stale file paths or symbol references in the body that no longer exist in the repo.

(Near-duplicate detection used to live here as a one-liner. It now has its own dimension 6 — it's a relationship-between-tickets concern, not a per-ticket text concern, and it deserves real detection criteria.)

5. Codebase relevance — the flagship dimension

For each ticket — particularly those labeled proposed, ready, in-progress, or blocked — judge whether the work the ticket describes is still real, given the current state of the code.

Process per ticket:

1. Read Problem + Approach + Acceptance Criteria.
2. Identify the artifacts the ticket touches: file paths, module names, feature names, types, functions, routes, env vars, UI surfaces, data fields.
3. For each artifact, check current state via Grep / Glob / Read:

• Does it still exist? (Stale if not.)
• Has its shape changed enough that the ticket no longer applies?

1. Read git log since the ticket's creation date for commits touching the relevant paths. Skim the diffs.
2. For each acceptance criterion, judge whether current code satisfies it. Be specific — cite files and lines.
3. Land on exactly one verdict:

Evidence rule: vague claims like "looks done" don't count. Every verdict cites specific files, line numbers, and commits.

Scaling: if the repo has many open tickets (say, >50), prioritize this dimension for in-progress, then blocked, then ready, then proposed. Run dimensions 1–4 across all tickets regardless. If the user asked for a quick pass, say so in the report and list which tickets weren't deeply examined.

6. Cross-ticket duplication & overlap

Two tickets that describe essentially the same work surface waste backlog real estate, scatter related context, and lead to one being silently-completed by the other. Title-identical detection (does any other ticket have the literal same title?) catches only the easy cases. The hard cases are tickets with different titles, sometimes different Groups, that nonetheless cover the same change — those are what this dimension targets.

Run this dimension after dimension 5. Dimension 5's verdicts feed in: a duplicate of a done/closed ticket is the silently-completed half, which is a critical finding.

Vocabulary

Detection signals

Field names below follow the project's schema — the examples use Airtable-flavored names (Scope, Deliverable, Group, Detected by). When auditing another source, map to its equivalents (canonical schema: Problem, Approach, Acceptance Criteria, thematic label), same as dimension 1's concept-over-name rule.

For each candidate pair (or cluster), score on:

1. Same `Group` / area / theme. Gating filter — only compare tickets in the same Group. Cross-group comparisons explode the search space and rarely surface real duplicates.
2. Touched-file overlap. Extract file paths from Scope, Deliverable, and Acceptance. >50% shared paths is a strong signal.
3. Title fuzzy match. Word-overlap after stripping stopwords ("add", "fix", "implement", "wire", "update"). >50% content-word overlap is a strong signal.
4. Scope-text semantic overlap. Read both Scopes. Are they describing the same change? This is the judgment call that distinguishes duplicate from related-but-distinct — and the most expensive signal to compute.
5. Same `Detected by` source. If both say "REM-040", "wave review 2026-05-21", or "audit 2026-05-28", they may be the same observation filed twice.
6. Same `Acceptance` criteria. Literal overlap in checkbox text.

A pair is a duplicate if signal 1 (same Group) is true AND at least 2 of signals 2–6 are strong.

Cross-check against done/closed