ratacat/claude-skills/research

Deep Research Skill

A Graph-of-Thoughts research system that builds a knowledge graph through structured exploration, stores all findings in SQLite, and uses graph operations (merging, traversal, contradiction detection, relationship inference) to synthesize insights.

CRITICAL REQUIREMENTS

You MUST use the database for all operations. Every finding, every URL, every search query, every relationship MUST be recorded. The database is not optional—it is the core of this methodology. If you skip database operations, the research is invalid.

You MUST perform graph operations. After each exploration phase, you MUST run merging, relationship inference, and contradiction detection. These are not suggestions—they are required steps.

You MUST work iteratively. Do not try to gather all information then synthesize. Work in cycles: explore → store → analyze graph → identify gaps → explore gaps → repeat.

Arguments

Parse the user's input to extract:

• topic: The research subject (required unless --list or --continue)
• --depth N: How many levels deep to explore (default: 2)
• --branches N: How many aspects to explore in parallel (default: 5)
• --continue "topic": Resume a previous research session
• --list: Show all saved research sessions

Database Setup

Before ANY research, initialize the database. This is mandatory:

sqlite3 research.db < ~/.claude/skills/research/schema.sql 2>/dev/null || true

Verify the database exists and has tables:

sqlite3 research.db "SELECT name FROM sqlite_master WHERE type='table';"

If tables are missing, the research CANNOT proceed.

PHASE 1: SESSION INITIALIZATION

1.1 Create Session

INSERT INTO sessions (topic, depth_remaining, branches)
VALUES ('{topic}', {depth}, {branches});

Immediately retrieve and store the session ID:

SELECT last_insert_rowid();

Store this session_id. You will use it in EVERY subsequent database operation.

1.2 Decompose Topic into Branches

Break the topic into {branches} distinct aspects. For each aspect, create an initial "lead" finding:

INSERT INTO findings (session_id, content, relevance_score, finding_type, branch_name, depth_level)
VALUES
  ({session_id}, 'BRANCH: {aspect_1_description}', 8, 'lead', '{aspect_1_name}', 0),
  ({session_id}, 'BRANCH: {aspect_2_description}', 8, 'lead', '{aspect_2_name}', 0),
  -- ... for all branches
;

This seeds the graph with initial nodes to explore.

PHASE 2: EXPLORATION

For each unexplored lead (finding_type='lead' that hasn't been explored):

2.1 Generate Search Query

Create a focused search query for the lead. Record it BEFORE executing:

INSERT INTO search_queries (session_id, query, branch_name, depth_level)
VALUES ({session_id}, '{query}', '{branch}', {current_depth});

2.2 Execute Search

python3 ~/.claude/skills/brave-search/brave.py web "{query}" --count 10 --extra-snippets

2.3 Record URLs

For EVERY URL in results, record it BEFORE fetching:

INSERT OR IGNORE INTO explored_urls (session_id, url, status)
VALUES ({session_id}, '{url}', 'pending');

2.4 Fetch and Extract

For promising URLs (top 3-5), fetch content with WebFetch. After fetching, update status:

UPDATE explored_urls SET status = 'fetched' WHERE session_id = {session_id} AND url = '{url}';

If fetch fails:

UPDATE explored_urls SET status = 'failed' WHERE session_id = {session_id} AND url = '{url}';

2.5 Create Findings

For EACH discrete piece of information extracted, create a finding:

INSERT INTO findings (session_id, content, source_url, relevance_score, finding_type, branch_name, depth_level)
VALUES ({session_id}, '{content}', '{url}', {score}, '{type}', '{branch}', {depth});

Scoring Criteria (APPLY STRICTLY):

• 9-10: Directly answers a key question; novel insight; actionable
• 7-8: Useful context; supports or challenges other findings
• 4-6: Tangentially related; may be useful for completeness
• 1-3: Barely relevant; keep only if nothing better exists
• 0: Off-topic; do not insert

Finding Types:

• fact: Verifiable piece of information with source
• lead: Promising direction requiring further exploration
• question: Unresolved question raised by the research
• theme: Recurring pattern observed across multiple sources
• contradiction: Conflicting information (see Phase 3.3)

2.6 Update Search Query Result Count

UPDATE search_queries SET result_count = {count}
WHERE session_id = {session_id} AND query = '{query}';

PHASE 3: GRAPH OPERATIONS

These operations are MANDATORY after each exploration cycle. Do not skip them.

3.1 MERGING: Detect and Combine Similar Findings

Similar findings fragment the graph and reduce insight quality. You MUST merge them.

Step 1: Identify Candidates

Query findings in the same branch that may be duplicates:

SELECT f1.id as id1, f2.id as id2, f1.content as content1, f2.content as content2
FROM findings f1
JOIN findings f2 ON f1.session_id = f2.session_id
  AND f1.branch_name = f2.branch_name
  AND f1.id < f2.id
WHERE f1.session_id = {session_id}
  AND f1.finding_type = f2.finding_type;

Step 2: Evaluate Similarity

For each pair, assess semantic similarity. Findings should be merged if:

• They state the same fact with different wording
• They cite the same source for the same claim
• One is a subset of the other

Step 3: Execute Merge

When merging finding B into finding A (keeping A):

-- Update A with combined content and best score
UPDATE findings
SET content = '{merged_content}',
    relevance_score = MAX(relevance_score, {b_score}),
    source_url = COALESCE(source_url, '{b_url}')
WHERE id = {a_id};
 
-- Transfer all relationships from B to A
UPDATE relationships SET from_finding_id = {a_id} WHERE from_finding_id = {b_id};
UPDATE relationships SET to_finding_id = {a_id} WHERE to_finding_id = {b_id};
 
-- Transfer entity links
UPDATE finding_entities SET finding_id = {a_id} WHERE finding_id = {b_id};
 
-- Delete B
DELETE FROM findings WHERE id = {b_id};

Step 4: Record Merge

Log the merge for audit trail (add to content or create a merge log table if needed).

3.2 RELATIONSHIP INFERENCE: Build the Graph Edges

Findings without relationships are isolated nodes. You MUST connect them.

Relationship Types:

• supports: Finding A provides evidence for Finding B
• contradicts: Finding A conflicts with Finding B
• elaborates: Finding A provides additional detail about Finding B
• related_to: Finding A and B discuss the same subtopic

Step 1: Query Unconnected Findings

SELECT f.id, f.content, f.branch_name, f.finding_type
FROM findings f
WHERE f.session_id = {session_id}
  AND f.id NOT IN (SELECT from_finding_id FROM relationships)
  AND f.id NOT IN (SELECT to_finding_id FROM relationships)
  AND f.finding_type IN ('fact', 'theme');

Step 2: For Each Unconnected Finding, Find Related Findings

SELECT id, content, branch_name FROM findings
WHERE session_id = {session_id}
  AND id != {current_id}
  AND finding_type IN ('fact', 'theme', 'lead');

Step 3: Evaluate and Create Relationships

For each potential pair, determine if a relationship exists and its type. If yes:

INSERT INTO relationships (from_finding_id, to_finding_id, relationship_type)
VALUES ({from_id}, {to_id}, '{type}');

Inference Rules:

• If Finding A cites a study and Finding B cites the same study → related_to
• If Finding A states X and Finding B provides mechanism for X → supports
• If Finding A says "X works" and Finding B says "X doesn't work" → contradicts
• If Finding A is general and Finding B is specific case → elaborates

3.3 CONTRADICTION DETECTION

Contradictions are HIGH-VALUE findings. They indicate uncertainty, bias, or nuance.

Step 1: Query Potential Contradictions

SELECT f1.id, f1.content, f1.source_url, f2.id, f2.content, f2.source_url
FROM findings f1
JOIN findings f2 ON f1.session_id = f2.session_id AND f1.id < f2.id
WHERE f1.session_id = {session_id}
  AND f1.finding_type = 'fact'
  AND f2.finding_type = 'fact'
  AND f1.branch_name = f2.branch_name;

Step 2: Evaluate Each Pair

Look for:

• Opposite claims about effectiveness
• Conflicting statistics or numbers
• Different conclusions from same evidence
• Mutually exclusive recommendations

Step 3: Record Contradiction

-- Create relationship
INSERT INTO relationships (from_finding_id, to_finding_id, relationship_type)
VALUES ({id1}, {id2}, 'contradicts');
 
-- Create a contradiction finding
INSERT INTO findings (session_id, content, relevance_score, finding_type, branch_name, depth_level)
VALUES (
  {session_id},
  'CONTRADICTION: {description of conflict between findings}. Source 1: {url1}. Source 2: {url2}. Possible explanations: {analysis}',
  9,
  'contradiction',
  '{branch}',
  {depth}
);

Contradictions scoring 9+ because they reveal important nuances.

3.4 ENTITY EXTRACTION AND LINKING

Entities enable cross-session knowledge and pattern detection.

Step 1: Extract Entities from New Findings

For each new finding, identify named entities:

• People (researchers, doctors, patients in case studies)
• Organizations (hospitals, research institutions, companies)
• Concepts (specific treatments, mechanisms, conditions)
• Technologies (devices, drugs, protocols)

Step 2: Create or Find Entity

-- Try to insert (will fail silently if exists due to UNIQUE constraint)
INSERT OR IGNORE INTO entities (name, entity_type) VALUES ('{name}', '{type}');
 
-- Get the entity ID
SELECT id FROM entities WHERE name = '{name}';

Step 3: Link Finding to Entity

INSERT OR IGNORE INTO finding_entities (finding_id, entity_id)
VALUES ({finding_id}, {entity_id});

PHASE 4: GRAPH TRAVERSAL AND GAP ANALYSIS

After graph operations, analyze the graph structure to identify gaps.

4.1 Find Isolated Clusters

-- Findings with no relationships (isolated nodes)
SELECT id, content, branch_name FROM findings
WHERE session_id = {session_id}
  AND id NOT IN (SELECT from_finding_id FROM relationships)
  AND id NOT IN (SELECT to_finding_id FROM relationships);

Isolated nodes indicate unexplored connections. Create leads to investigate them.

4.2 Find Weakly Supported Claims

-- High-relevance findings with few supporting relationships
SELECT f.id, f.content, f.relevance_score, COUNT(r.id) as support_count
FROM findings f
LEFT JOIN relationships r ON r.to_finding_id = f.id AND r.relationship_type = 'supports'
WHERE f.session_id = {session_id} AND f.relevance_score >= 8
GROUP BY f.id
HAVING support_count < 2;

Important claims need corroboration. Create leads to find supporting evidence.

4.3 Find Unresolved Questions

SELECT id, content FROM findings
WHERE session_id = {session_id} AND finding_type = 'question';

Each question is a gap. Prioritize by relevance score.

4.4 Identify Unexplored Leads

SELECT f.id, f.content, f.branch_name
FROM findings f
LEFT JOIN search_queries sq ON sq.session_id = f.session_id
  AND sq.branch_name = f.branch_name
  AND sq.depth_level > f.depth_level
WHERE f.session_id = {session_id}
  AND f.finding_type = 'lead'
  AND f.relevance_score >= 7
  AND sq.id IS NULL;

These are leads that haven't been followed. If depth_remaining > 0, explore them.

4.5 Create New Leads from Gaps

For each gap identified, create a lead finding:

INSERT INTO findings (session_id, content, relevance_score, finding_type, branch_name, depth_level)
VALUES ({session_id}, 'GAP: {description}', 8, 'lead', '{branch}', {depth});

PHASE 5: RECURSIVE EXPLORATION

If depth_remaining > 0 and unexplored leads exist:

5.1 Decrement Depth