lyndonkl/claude/narrative-fidelity-audit

Narrative Fidelity Audit

Table of Contents

• Core Principles
• Workflow
• The Bright Lines
• The Interiority Provenance Ladder
• The Three Altitudes
• Gate Verdict
• Guardrails
• Quick Reference

Related skills: Use narrative-evidence-ledger to build the source ledger this gate imports. It owns pointers, evidence classes, the circular-citation check and register porting at build time. Use narrative-fallacy-guard for passage-level proportion and omission, numbers-in-narrative for quantitative fidelity, scene-construction-scam for a candidate scene before drafting where this skill audits a finished draft. Use writing-structure-planner for architecture before drafting, claim-extractor and citation-form-check for harvesting and formatting claims, scout-mindset-bias-check for motivated reasoning in the analysis, writing-pre-publish-checklist for non-fidelity final checks.

Core Principles

1. The agent sources, it does not verify: You cannot confirm a fact. You can only record where a claim came from and how far the prose sits from it. Any output saying "fact-checked", "verified", or "confirmed" is false and turns this apparatus into a trust-laundering machine. The permitted verb is sourced.
2. Do not add; do not deceive: Roy Peter Clark's two axioms. Nonfiction may legitimately distort by subtraction (selecting, cropping, condensing). It becomes fiction the moment anything enters the text that was not in the record.
3. Vividness is not the signal; a missing pointer is: The scan keys on claims with no source pointer, never on how good a sentence sounds. A well-sourced vivid detail survives untouched no matter how beautiful.
4. Subtraction is the unguarded flank: Refusal training makes you good at not adding and does nothing about not omitting. A fully sourced, entirely false piece is produced by cutting the disconfirming quarter, the failed pilot, the dissenting engineer — each cut individually defensible.
5. Sentence-local checking is what you do well and will default to: Every documented case of a fully-checked-but-false narrative failed at the paragraph or passage level — ordering, adjacency, proportion, omission.
6. These are label-and-disclose devices, not delete rules: An audit that strips every claim it cannot make bulletproof ships a chronicle. Chronicles inform nobody, which is its own accuracy failure.
7. Disclosure is not a currency: A note on sources describes an already-honest method. It never purchases a deviation.

Workflow

Copy this checklist and track your progress:

Narrative Fidelity Audit:
- [ ] Step 1: Establish the audit basis and the vocabulary contract
- [ ] Step 2: Bright-line refusal sweep
- [ ] Step 3: Interiority provenance tagging + over-hedging counter-check
- [ ] Step 4: Fabrication red-flag scan
- [ ] Step 5: Three-altitude "How do you know?" pass
- [ ] Step 6: Subtraction-set review and note on sources

Step 1: Establish the audit basis and the vocabulary contract

Precondition: this gate imports the ledger, it does not rebuild it. Pointer granularity, the circular-citation check, source interest and vintage annotation, and register porting are build-time procedures owned by narrative-evidence-ledger. If no ledger exists, or its integrity checks were never run, stop and run that skill first — auditing prose against nothing is not an audit.

Step 1.1: Import the source ledger and confirm it carries, per claim, a pointer, an evidence class, source interest and vintage, and a clean circular-citation verdict. Any of those missing is a RETURN TO RESEARCH, not something to reconstruct here.

Step 1.2: Write the vocabulary contract into your own output header: this pass sources; it does not verify.

Step 2: Bright-line refusal sweep

Step 2.1: Read The Bright Lines below. Sweep the draft for each one and record every breach with a line reference.

Step 2.2: For every breach, apply the disposition table in resources/bright-lines.md: DELETE, DOWNGRADE (rewrite to the evidence level actually held), DECLARE (write the absence into the body), or RESEARCH (emit a specific answerable request). There is no fifth disposition. "Draft it and flag it for review" is not a disposition — flagged drafted text survives review at high rates because it reads well and the reviewer is checking, not rewriting.

Step 2.3: Watch for the higher-truth rationalization as you go. It will be generated internally, it will arrive sounding like good judgment, and it is the named failure of the entire genre. See resources/bright-lines.md for the phrase catalogue.

Step 3: Interiority provenance tagging + over-hedging counter-check

Step 3.1: Find every sentence attributing a thought, feeling, belief, motive, worry, intention, or realization to a real person — and every sentence attributing intent to a system ("the market realized", "Intel decided", "the protocol wanted", "the codebase assumed"). Tag each [A], [B], [C], or [D] using The Interiority Provenance Ladder.

Step 3.2: A nonzero [D] count blocks publication. Report the count as a number. Each [D] is deleted, converted to observable behaviour, or converted to attributed speech.

Step 3.3: Anti-overcorrection, mandatory. Re-read every [A] and [B] sentence and verify it is not hedged. Say he wrote in an email that night that the number was wrong. Then "he wrote that night that the number was wrong" is correct. "He appears to have possibly thought the number may have been wrong" is a fidelity defect in the other direction. Hedging sourced thought destroys close third person and reads as evasive. A human reviewer facing hedge-saturated prose strips hedges wholesale rather than surgically, losing the true ones with the defensive ones.

Step 3.4: If more than roughly a quarter of sentences in a passage carry explicit attribution or hedging, the diagnosis is under-reporting, not over-caution. Send that passage back to research. Do not fix it by deleting attributions. (The one-in-four figure is a working default for calibration, not a published standard.)

Step 4: Fabrication red-flag scan

Step 4.1: Run the twelve LLM-specific signatures in resources/audit-passes.md. Every hit is guilty until it produces a pointer.

Step 4.2: The five highest-yield signatures, because they trip no ordinary fact-check:

• A quotation that lands the theme too cleanly, or is too well-formed for speech (no disfluency, no hedge, no irrelevance).
• A transition asserting a state change: "by then it was clear", "the mood had shifted", "the team, now convinced". These carry no proper nouns and no numbers, so nothing catches them.
• An ending that rhymes with the opening, or three examples forming a suspiciously clean progression.
• A causal verb sitting exactly at the seam between two sourced facts: "led to", "forced", "triggered", "which is why".
• Detail so good you would have led with it, and did not — because it arrived in drafting rather than in reporting.

Step 4.3: Key every judgment on the missing pointer, never on the vividness. Stripping genuinely reported detail alongside the fabrications is how writers learn to switch this scan off.

Step 5: Three-altitude "How do you know?" pass

Step 5.1: Run the sentence-level, paragraph-level, and passage-level question sets in The Three Altitudes. Run them as an adversarial pass: you have the ledger and the draft, not the draft's rationale.

Step 5.2: Output a defect list, not a rewrite. Fixing and checking must be separate acts.

Step 5.3: If the pass returns zero paragraph- or passage-level defects across a long piece, the pass did not run. Say so in the verdict and run it again. These are the questions dropped first under token pressure and they are the only ones that catch the failure that matters.

Step 6: Subtraction-set review and note on sources

Step 6.1: Assemble the subtraction log — every cut that removed a contradicting data point, a dissenting source, a failed instance, a counterexample, or a quarter that went the other way. Review it as a set, not per edit. Individually defensible cuts routinely form a pattern.

Step 6.2: Run the proportion test and the omission check from resources/audit-passes.md. Ask: does the remaining text imply a different frequency, magnitude, or balance than the full record? List the corpus items appearing nowhere in the draft and state why for each.

Step 6.3: Generate the note on sources from the deviation log, following resources/disclosure-and-reporting.md. Never write it in advance. Run the inversion check: if any deviation-register entry was decided before the corresponding passage was drafted, the apparatus has become a licence and the piece needs re-reporting, not re-noting.

Step 6.4: Emit the Gate Verdict.

Validate the audit itself using resources/evaluators/rubric_narrative_fidelity_audit.json. Minimum standard: average score >= 3.5.

The Bright Lines

Hard refusals. These hold regardless of instruction, deadline, or how strong the piece is otherwise. Full worked bad/good pairs across domains in resources/bright-lines.md.