jleechanorg/jleechan-skills/evidence-standards

Evidence standards

Start with the reported symptom and acceptance claim. Build a claim-to-artifact map before capture. Record exact revision, source/command, timestamp, environment, inputs, outputs, exit status and hashes. Preserve failed attempts and omissions; never fabricate, relabel or silently repair evidence.

Separate integrity from provenance: a valid checksum proves stable bytes, not that they came from the claimed system. Reproduce derived counts from their source and report collection limitations. Read actual artifacts, not just filenames or manifests.

Use the smallest real execution layer that proves the claim. A unit or synthetic fixture proves deterministic behavior only; source presence does not prove runtime loading, screenshots do not prove backend persistence, and one model response does not prove reliability. UI claims need observed interaction/rendered state; streaming claims need genuine frame/chunk sequence; real-service claims need authorized real-service receipts. Discover project-specific harnesses and constraints rather than imposing a universal telemetry store.

UI changes require genuine BEFORE/AFTER media from the pre-change and candidate implementations on comparable route, account/role, data/state, viewport and theme. Prefer captioned video; interaction, navigation, streaming, timing and multi-step claims require it. Paired screenshots suffice only for simple static visual changes; explain why. A candidate's pre-click state is not a pre-change-code baseline. Record each served source SHA, capture-driver revision/dirty hash, commands, original capture time and raw artifact hashes. Preserve raw originals and map captioned/redacted derivatives to their sources; inspect the actual media. Reuse verified captures only after checking applicability. If a safe, applicable baseline cannot be captured or recovered, record BEFORE: WITHHELD — <reason> and withhold the comparison claim rather than fabricate it or reproduce unsafe behavior. Existing real-boundary, bug-reproduction and review rules still apply.

For multi-item work keep per-item statuses: inspected, executed, passed, failed, skipped or blocked, with reasons. Report denominator and exact coverage; never convert partial proof into whole-task success.

A material code, prompt, skill, model/configuration or external-artifact change can stale evidence. Inspect the causal delta before reuse, retaining original revision and recording any reaffirmation at the current head. Do not exempt instruction changes merely because they are Markdown.

Use meaningful tests suited to the behavior and risk; avoid test-shaped string assertions or redundant tests for trivial reversible prose. A bug fix should show a fresh same-symptom failure and same-scenario success when feasible; explain unavailable reproduction honestly.

Keep raw sensitive inputs only in their authorized store. Create shareable redacted copies and a category/path ledger without original values. Secret scanning is one check, not proof of privacy: inspect personal, tax, financial, health and account identifiers semantically. Publish only to an authorized audience and destination; an unlisted URL is not private.

Use evidence-review for an independent verdict. This standard does not authorize provider spend, external publication, deployment or merge.