danielmiessler/lifeos/detectai

DetectAI

Answers two questions, how much does this read as machine-generated? and does it carry an embedded mark?, with four independent measures: a heuristic audit against a severity-tiered pattern catalog, which explains why each flag fired; deterministic statistical signals (LIFEOS/TOOLS/StatSignals.ts), each labeled with its reliability; an empirical Pangram score, which costs money and needs a key; and a keyless watermark scan for covert channels in the bytes. Detect, the statistics and Watermark are free and need no key. The measures disagree often, and the disagreement is the useful part: text can clear every pattern on the list and still score 100% AI, which means the tells are structural rather than lexical, and a watermark hit is bytes-level proof whatever the other measures say.

Ideal State

• Every answer rests on measures that actually ran (the pattern catalog, StatSignals.ts, a recorded Pangram run, WatermarkScan.ts), and the report names which ran and which did not.
• A plain "is this AI?" with a key configured gets the heuristic audit and the Pangram score side by side, and where they disagree the report says what the gap means.
• No report treats one signal as proof of authorship: scores carry their length and baseline caveats, and a clean watermark scan is never read as "human-written".
• The text under review is reported on, never rewritten.

Intent Routing

"Is this watermarked?" routes to Watermark, which answers a different question: whether a covert channel is embedded in the bytes, not whether the prose reads as AI. For "is this AI and is it marked?", run Score plus Watermark.

Data & Config

Detect and Watermark need no setup. Score needs PANGRAM_API_KEY in ~/.claude/.env (prepaid credits, about $0.05 per 1,000 words); key, credits, verification and troubleshooting are in Setup.md.

Gotchas

• Never degrade writing to beat a detector. Injected typos, broken sentences, and "humanizer" laundering damage the prose and don't fix the underlying problem. If text must read human, the fix is a human in the loop.

Examples

• "Scan this post for AI tells, don't change it" → Detect: tiered P0/P1/P2 flag report, each marked clear-problem vs judgment-call, no edits.
• "Is this AI generated?" → Score: single Pangram run, headline verdict plus AI/AI-assisted/human percentages, with the length caveat stated if the sample is short.
• "Which of these three drafts reads most human?" → Score in batch-compare mode: one call per draft, ranked table, relative comparison foregrounded over absolute numbers.
• "Did my rewrite actually help?" → Score before and after, with two known-human passages scored in the same batch as calibration.