kaichen/agent-skillset/codebase-fact-check

Codebase Fact Check

Purpose

Produce answers that are grounded in the actual repository instead of product assumptions, docs-only reasoning, or visible UI labels. The user's repeated pattern is to ask for the real source path, real control fields, and real activation conditions before deciding whether to patch.

When to use

Use this skill for:

• Codebase exploration, architecture tracing, source-backed explanations, PR/spec reviews, and bug triage.
• Questions that compare a visible symptom with implementation reality, such as model labels, Slack threading, runtime ownership, or route behavior.
• Review requests where findings must be actionable and tied to exact locations.

Do not use it for pure brainstorming, copy editing, or tasks where the user explicitly says not to inspect files.

Workflow

1. Orient on the repo

• Run pwd and inspect the nearest AGENTS.md.
• Use rg --files or fd to find likely entrypoints, tests, config, and docs.
• Read the smallest set of files that can answer the question.

1. Trace the primary path

• Identify the first real entrypoint: CLI command, route handler, monitor, hook, service, component, or worker.
• Follow data and control flow to the final side effect.
• Name the decisive symbols: functions, classes, fields, env vars, config keys, routes, and storage tables.

1. Separate facts from hypotheses

• Mark as fact only what is proven by source, local files, tests, logs, or a direct command.
• Treat UI labels, model names, stale docs, and memory summaries as clues until source confirms them.
• If external docs are needed because the code calls a third-party API, use primary docs and cite them.

1. Check for real enablement

• Distinguish "code exists" from "this path is active."
• Verify wiring: imports, route registration, package scripts, feature flags, env gates, config defaults, and runtime command path.
• For reviews, include missing tests only when they would catch the actual risk.

1. Report with evidence

• Findings first for reviews, ordered by severity.
• For explanations, start with the direct answer, then show the source path that proves it.
• Use exact local file links with line numbers when available.

Review Output

For review-style tasks, use:

**Findings**
-[P1] Title
  File: path:line
  Why it matters:
  Fix:
 
**Open Questions**
-...
 
**Verification**
-Command/result or why not run.

For explanation-style tasks, use:

Direct answer:
 
Evidence:
-path:line -> what it proves
-path:line -> what it proves
 
What could go wrong:
-...

Common Failure Modes

Key Principle

The answer is not "what seems likely"; the answer is the smallest source-backed path that explains the behavior.