jwynia/agent-skills/ebook-analysis

Ebook Analysis: Non-Fiction Knowledge Extraction

You analyze ebooks to extract knowledge with full citation traceability. This skill supports two complementary extraction modes:

1. Concept Extraction - Extract ideas classified by abstraction (principle → tactic)
2. Entity Extraction - Extract named things (studies, researchers, frameworks, anecdotes) that persist across books

Core Principle

Every extraction must be traceable to its exact source. Citation traceability is non-negotiable. Extract less with full provenance rather than more without it.

Two Extraction Modes

Mode 1: Concept Extraction

For extracting IDEAS organized by abstraction level.

Use when: Analyzing a book for transferable ideas, building a concept taxonomy, understanding how abstract principles relate to concrete tactics.

Output: JSON files (analysis.json, concepts.json)

Example: "Spaced repetition improves retention" is a MECHANISM at Layer 2.

Mode 2: Entity Extraction

For extracting NAMED THINGS that can be cross-referenced across books.

Use when: Building a knowledge base where the same study, researcher, or framework appears in multiple books. The goal is entity resolution—recognizing that "Hogarth's framework" in Range is the same as "kind/wicked environments" mentioned elsewhere.

Output: Markdown files in knowledge base structure

Example: "Kind vs Wicked Environments" is a FRAMEWORK by Robin Hogarth.

Choosing a Mode

Entity Extraction Mode (Detailed)

Entity Types

Extended Entity Type Guidance

Some entities don't fit cleanly into the five types. Guidelines:

When uncertain: Default to anecdote for narratives/events, concept for ideas, framework for systematic methods.

Author-as-Subject Pattern

When the book's author is also a significant entity (e.g., Jane McGonigal in Imaginable):

Create a researcher entity if:

• Author has notable prior work or institutional affiliation
• Author appears in Wikipedia or other reference sources
• Author's background/credentials are relevant to understanding the book
• Other books in your collection might reference them

Skip if:

• Author is primarily known only for this book
• No external sources to verify/enrich the entity

Template addition for author-subjects:

## Note
This researcher is the author of [Book] in our collection. Their frameworks and concepts are documented separately.

Entity File Template

# [Entity Name]
**Type:** study | researcher | framework | anecdote | concept
**Status:** stub | partial | solid | authoritative
**Last Updated:** YYYY-MM-DD
**Aliases:** alias1, alias2, alias3
 
## Summary
[2-3 sentence synthesized understanding]
 
## Key Findings / What It Illustrates
1.[Claim or finding with source]
   — Source: [Book], Ch.[X]
 
2.[Another claim]
   — Source: [Book], Ch.[X]
 
## Key Quotes
> "Quotable text here."
 
> "Another memorable quote."
 
## Sources in Collection
| Book | Author | How It's Used | Citation |
|------|--------|---------------|----------|
| Range | Epstein | [Role in book] | Ch.X |
 
## Sources NOT in Collection
-[Book that would enrich this entity]
 
## Related Entities
-[Other Entity](../type/other-entity.md) - Relationship description
 
## Open Questions
-[What we don't yet know]

Knowledge Base Structure

/knowledge/
├── _index.md                    # Master registry
├── _entities.json               # Searchable index (generated)
│
├── nonfiction/
│   ├── _index.md                # Domain index
│   ├── _[book]-quotes.md        # Book-specific quotes file
│   ├── studies/
│   │   ├── flynn-effect.md
│   │   └── chase-simon-chunking.md
│   ├── researchers/
│   │   ├── hogarth-robin.md
│   │   └── tetlock-philip.md
│   ├── frameworks/
│   │   ├── kind-vs-wicked-environments.md
│   │   └── desirable-difficulties.md
│   ├── anecdotes/
│   │   ├── tiger-vs-roger.md
│   │   └── challenger-disaster.md
│   └── concepts/
│       ├── cognitive-entrenchment.md
│       └── match-quality.md
│
├── cooking/                     # Domain-specific structure
│   ├── techniques/
│   ├── ingredients/
│   └── equipment/
│
└── technical/
    ├── patterns/
    └── technologies/

Quotes Extraction

Quotable quotes are a distinct extraction type. For each book, create a quotes file:

File: _[book-slug]-quotes.md

Structure:

# Quotable Quotes from [Book Title]
**Author:** [Author]
**Last Updated:** YYYY-MM-DD
 
## On [Theme 1]
> "Quote text here."
 
> "Another quote on same theme."
 
## On [Theme 2]
> "Quote on different theme."

What makes a good quote:

• Memorable phrasing that captures a key insight
• Self-contained (understandable without context)
• Surprising or counterintuitive formulation
• Useful for presentations, writing, or reference

Entity Extraction Workflow

1. Scan book - Read through identifying named studies, researchers, frameworks, illustrative stories
2. Check existing entities - Use kb-resolve-entity.ts to see if entity already exists
3. Create or update - New entity → create file; existing → add as source
4. Add quotes - Extract memorable quotes to quotes file
5. Cross-link - Add Related Entities sections
6. Regenerate index - Run kb-generate-index.ts

Entity Extraction States (KB0-KB5)

Cross-Book Synthesis Workflow

Triggered when: 2+ books have been extracted to the knowledge base.

Goals:

1. Find entities that appear in multiple books
2. Identify conceptual connections between books
3. Surface contradictions or complementary perspectives
4. Update entity files with multi-source synthesis

Process:

1. Entity overlap detection

``bash # Find entities with 2+ sources grep -l "Sources in Collection" knowledge/nonfiction/**/*.md | \ xargs grep -l "| .* | .* |" | head -20 `` Or manually review entities updated with new source.

1. Conceptual connection mapping

• Compare frameworks across books (e.g., Range's "wicked environments" ↔ Imaginable's "futures thinking")
• Identify shared researchers (e.g., Tetlock appears in both Range and Imaginable)
• Look for complementary themes (prediction failure → preparation despite uncertainty)

1. Synthesis documentation

For entities appearing in 2+ books, update the Summary section: ``markdown ## Summary [Synthesized understanding from BOTH sources, noting agreements and differences] ``

1. Cross-book insights

Document thematic connections in context/insights/cross-book-{theme}.md: ```markdown # Cross-Book Insight: [Theme]

## Books Contributing

• Range (Epstein) - [perspective]
• Imaginable (McGonigal) - [perspective]

## Synthesis [How the books complement or contradict each other] ```

Concept Extraction Mode (Detailed)

Concept Types (Abstract → Concrete)

Abstraction Layers

Relationship Types

Concept Extraction States (EA0-EA7)

Concept Extraction Workflow

1. Parse - Run ea-parse.ts to chunk book with position tracking
2. Extract - Present chunks to LLM for concept identification with exact quotes
3. Classify - Assign type (principle→tactic) and layer (0-4)
4. Annotate - Add themes and functional analysis
5. Link - Connect related concepts
6. Export - Generate analysis.json, concepts.json, report.md

Available Tools

Parsing Tools

ea-parse.ts

Parse ebook files into chunks with metadata and position tracking.

deno run --allow-read scripts/ea-parse.ts path/to/book.txt
deno run --allow-read scripts/ea-parse.ts path/to/book.epub --format epub
deno run --allow-read scripts/ea-parse.ts book.txt --chunk-size 1500 --overlap 150

Output: JSON with metadata, chapters (if detected), and chunks with positions.

Knowledge Base Tools

kb-generate-index.ts

Scan knowledge base and generate searchable entity index.

deno run --allow-read --allow-write scripts/kb-generate-index.ts /path/to/knowledge

Output: Creates _entities.json with all entities, aliases, and metadata.