zby/llm-do/arscontexta-review

Runtime Configuration (Step 0 — before any processing)

Read these files to configure domain-specific behavior:

1. `arscontexta/ops/derivation-manifest.md` — vocabulary mapping, platform hints

• Use vocabulary.notes for the notes folder name
• Use vocabulary.note / vocabulary.note_plural for note type references
• Use vocabulary.verify for the process verb in output
• Use vocabulary.topic_map for MOC references
• Use vocabulary.templates for the templates folder path
• Use vocabulary.cmd_reflect for redirect when missing connections found

1. `arscontexta/ops/config.yaml` — processing depth, verification settings

• processing.depth: deep | standard | quick
• processing.verification.description_test: true | false
• processing.verification.schema_check: true | false
• processing.verification.link_check: true | false

If these files don't exist, use universal defaults.

Processing depth adaptation:

EXECUTE NOW

Target: $ARGUMENTS

Parse immediately:

• If target contains a note name: verify that specific note
• If target contains --handoff: output RALPH HANDOFF block at end
• If target is "all" or "recent": verify recently created/modified notes
• If target is empty: ask which note to verify

Anti-Shortcut Warning

Before marking verification as passed, you MUST complete ALL four categories:

1. COMPLETE description quality test — cold-read the title + description,

predict what the note contains, compare against actual content. A description that merely restates the title FAILS.

1. COMPLETE schema validation — check ALL required fields from the

template schema, verify ALL enum values are valid, confirm ALL constraints are met. A single missing required field FAILS.

1. COMPLETE link verification — confirm ALL wiki links in the note

resolve to existing files. A single dangling link FAILS.

1. COMPLETE index integration — verify the note appears in at least

one index's Core Ideas section with a context phrase. A note with no index mention FAILS.

Do NOT declare success after checking only one or two categories. ALL FOUR must pass.

Execute these steps IN ORDER:

Step 0: INDEX FRESHNESS CHECK

Before any retrieval tests, verify the semantic search index is current:

1. Try mcp__qmd__vsearch with a simple test query to confirm MCP availability
2. If MCP is available, proceed to Step 1
3. If MCP is unavailable (tool fails or returns error): note "retrieval test will be deferred" and proceed — do NOT let index issues block verification

The index freshness check prevents false retrieval failures on recently created notes. If the index is stale, retrieval test results should be interpreted with that context.

Step 1: RECITE (cold-read prediction test)

CRITICAL: Do NOT read the full note yet. Only read frontmatter.

This step tests whether the title + description alone enable an agent to predict the note's content. The cold-read constraint is the entire point — reading the note first contaminates the prediction.

1. Read ONLY title + description

Use Read with a line limit to get just the first few lines of frontmatter. Extract:

• Title (the filename without .md)
• Description (the description field)

Do NOT scroll past the frontmatter closing ---.

2. Form prediction

Before reading further, write out what you expect:

• Core argument: what claim does this note make?
• Mechanism: what reasoning or evidence does it use?
• Scope: what boundaries does the argument have?
• Likely connections: what other notes would it reference?

Write this prediction explicitly in your output. It must be specific enough to be wrong.

3. Read full note content

NOW read the complete note. Compare against your prediction.

4. Score prediction accuracy (1-5)

Passing threshold: 3 or above.

5. Run semantic retrieval test

Test whether the description enables semantic retrieval:

• Try mcp__qmd__vsearch with query = "[the note's description text]", collection = "docs-notes", limit = 10
• If MCP unavailable: report "retrieval test deferred (semantic search unavailable)" — do NOT skip silently

Check where the note appears in results:

• Top 3: description works well for semantic retrieval
• Position 4-10: adequate but could improve
• Not in top 10: flag — description may not convey the note's meaning

Why vsearch specifically: Agents find notes via semantic search during reflect and reweave. Testing with keyword search tests the wrong retrieval method. Full hybrid search with LLM reranking compensates for weak descriptions — too lenient. vsearch tests real semantic findability without hiding bad descriptions behind reranking.

6. Draft improved description if needed

If prediction score < 3:

• Diagnose the failure: too vague? missing mechanism? wrong emphasis? restates title?
• Draft an improved description that would score higher
• If you have Edit tool access: apply the improvement

7. Combined scoring

Step 2: VALIDATE (schema check)

Read the template that applies to this note type. Determine the template by checking:

• Note location (e.g., docs/notes/ uses the standard note template)
• Type field in frontmatter (if present, may indicate a specialized template)

If the vault has templates with _schema blocks, read the _schema from the relevant template for authoritative field requirements. If no _schema exists, use the checks below as defaults.

Required fields (FAIL if missing):

Description constraints (WARN if violated):

How to check "adds new info": Read the title, read the description. If the description says the same thing in different words, it fails this check. A good description adds: mechanism (how/why), scope (boundaries), implication (what follows), or context (where it applies).

YAML validity (FAIL if broken):

Domain-specific field enums (WARN if invalid):

If the note has fields with enumerated values (type, category, status, etc.), check them against the template's _schema.enums block. Each invalid enum value produces a WARN.

Relevant notes format (WARN if incorrect):

Topics format (FAIL if invalid):

Composability (WARN if fails):

Step 3: REVIEW (per-note health checks)

Run these 5 checks on the note:

1. YAML frontmatter integrity

• File starts with ---, has closing ---
• YAML parses without errors
• No duplicate keys

2. Description quality (independent of recite)

• Description is present and non-empty
• Description adds information beyond the title
• Description is not just the title rephrased

3. index connection

• Note appears in at least one index's Core Ideas section
• How to check: grep for [[note title]] in files that serve as indexs
• The note's Topics footer references a valid index
• A note with no index mention is orphaned — FAIL

4. Wiki link density

• Count outgoing wiki links in the note body (not just frontmatter)
• Expected minimum: 2 outgoing links
• If < 2: flag as sparse — the note is not participating in the graph
• Sparse notes should be routed to /connect for connection finding

5. Link resolution

• Scan ALL wiki links in the note — body, frontmatter relevant_notes, and Topics
• For each [[link]], confirm a matching file exists in the vault
• Exclude wiki links inside backtick-wrapped code blocks (single backtick or triple backtick) — these are syntax examples, not real links
• A single dangling link = FAIL with the specific broken link identified

Deep-only checks (when processing.depth = deep):

6. Orphan risk assessment

• Count incoming links: grep for [[note title]] across all .md files
• If 0 incoming links: AT RISK — note exists but nothing references it
• If 1 incoming link: LOW RISK — single point of connection
• If 2+ incoming links: OK

7. Content staleness detection

• Read the note's content and assess whether claims still seem current
• Check if referenced concepts/tools/approaches have changed
• Flag anything that reads as potentially outdated

8. Bundling analysis

• Does the note make multiple distinct claims that could be separate notes?
• Check: could you link to part of this note without dragging unrelated context?
• If yes: flag for potential splitting

Step 4: APPLY FIXES

If you have Edit tool access, apply fixes for clear-cut issues:

Auto-fix (safe to apply):

• Improved description if recite score < 3
• Missing --- frontmatter delimiters
• Trailing period on description
• Missing Topics footer (if obvious which index applies)

Do NOT auto-fix (requires judgment):

• Bundled notes (splitting requires understanding the claims)
• Content staleness (needs human review of factual accuracy)
• Missing connections (use /connect instead — connection finding is its own phase)
• Ambiguous index assignment (when note could fit multiple)

Step 5: Compile Results

Combine all checks into a unified report:

=== VERIFY: [note title] ===
 
RECITE:
  Prediction score: N/5
  Retrieval rank: #N (or "not in top 10" or "deferred")
  Description: [pass/improved/needs work]