mycelium-hq/ai-brain-starter/graphify

/graphify

`{SKILL_DIR}` = this skill's own folder (locally: the directory this SKILL.md lives in; a served brain substitutes the real absolute path before you read this). If a path does not resolve, name the missing file and stop — never guess another location.

Turn any folder of files into a navigable knowledge graph with community detection, an honest audit trail, and three outputs: interactive HTML, GraphRAG-ready JSON, and a plain-language GRAPH_REPORT.md.

⚡ Before running on a corpus larger than ~50 files, READ [OPTIMIZATIONS.md](./OPTIMIZATIONS.md). The wrapper scripts in scripts/ (dedupe, regex preflight, word-balanced chunking, label canonicalization, cache integration) typically cut LLM token cost by 80–92% and produce a higher-quality graph. The single most important step is calling graphify_canonicalize.py --cache after merging — without it, the next --update run repays the entire cost. Skip these wrappers and a 1,500-file vault will burn ~10M LLM tokens for the same graph that costs ~1M with them.

Usage

/graphify                                             # full pipeline on current directory → Obsidian vault
/graphify <path>                                      # full pipeline on specific path
/graphify <path> --mode deep                          # thorough extraction, richer INFERRED edges
/graphify <path> --update                             # incremental - re-extract only new/changed files
/graphify <path> --directed                            # build directed graph (preserves edge direction: source→target)
/graphify <path> --whisper-model medium                # use a larger Whisper model for better transcription accuracy (base|small|medium|large)
/graphify <path> --cluster-only                       # rerun clustering on existing graph
/graphify <path> --no-viz                             # skip visualization, just report + JSON
/graphify <path> --html                               # (HTML is generated by default - this flag is a no-op)
/graphify <path> --svg                                # also export graph.svg (embeds in Notion, GitHub)
/graphify <path> --graphml                            # export graph.graphml (Gephi, yEd)
/graphify <path> --neo4j                              # generate graphify-out/cypher.txt for Neo4j
/graphify <path> --neo4j-push bolt://localhost:7687   # push directly to Neo4j
/graphify <path> --mcp                                # start MCP stdio server for agent access
/graphify <path> --watch                              # watch folder, auto-rebuild on code changes (no LLM needed)
/graphify <path> --wiki                               # build agent-crawlable wiki (index.md + one article per community)
/graphify <path> --obsidian --obsidian-dir ~/vaults/my-project  # write vault to custom path (e.g. existing vault)
/graphify add <url>                                   # fetch URL, save to ./raw, update graph
/graphify add <url> --author "Name"                   # tag who wrote it
/graphify add <url> --contributor "Name"              # tag who added it to the corpus
/graphify query "<question>"                          # BFS traversal - broad context
/graphify query "<question>" --dfs                    # DFS - trace a specific path
/graphify query "<question>" --budget 1500            # cap answer at N tokens
/graphify path "AuthModule" "Database"                # shortest path between two concepts
/graphify explain "SwinTransformer"                   # plain-language explanation of a node

What graphify is for

graphify is built around Andrej Karpathy's /raw folder workflow: drop anything into a folder - papers, tweets, screenshots, code, notes - and get a structured knowledge graph that shows you what you didn't know was connected.

Three things it does that Claude alone cannot:

1. Persistent graph - relationships are stored in graphify-out/graph.json and survive across sessions. Ask questions weeks later without re-reading everything.
2. Honest audit trail - every edge is tagged EXTRACTED, INFERRED, or AMBIGUOUS. You know what was found vs invented.
3. Cross-document surprise - community detection finds connections between concepts in different files that you would never think to ask about directly.

Use it for:

• A codebase you're new to (understand architecture before touching anything)
• A reading list (papers + tweets + notes → one navigable graph)
• A research corpus (citation graph + concept graph in one)
• Your personal /raw folder (drop everything in, let it grow, query it)

CRITICAL: graph.json edge key

graphify.export.to_json() uses networkx's node_link_data format. Edges are stored under the key `"links"`, NOT "edges". If you ever write a custom script that loads graph.json and reads edges directly, use:

edges = graph.get('links', graph.get('edges', []))  # 'links' first — 'edges' is never set by to_json

Failing to do this gives you 0 edges silently and will corrupt any merge.

The skill's own pipeline uses internal temp files (.graphify_*.json) that DO use "edges" — that is fine because those files are written/read by the same code. This warning only applies to loading `graph.json` directly.

What You Must Do When Invoked

If no path was given, use . (current directory). Do not ask the user for a path.

Follow these steps in order. Do not skip steps.

Step 1 - Ensure graphify is installed

# Detect the correct Python interpreter (handles pipx, venv, system installs)
GRAPHIFY_BIN=$(which graphify 2>/dev/null)
if [ -n "$GRAPHIFY_BIN" ]; then
    PYTHON=$(head -1 "$GRAPHIFY_BIN" | tr -d '#!')
    case "$PYTHON" in
        *[!a-zA-Z0-9/_.-]*) PYTHON="python3" ;;
    esac
else
    PYTHON="python3"
fi
mkdir -p graphify-out
 
# Install path with real error surfacing. On failure, STOP and tell the user —
# do not continue with a broken interpreter. Historical bug: pip errors were
# swallowed by tail -3 + silent || chaining and the skill kept going.
if ! "$PYTHON" -c "import graphify" 2>/dev/null; then
    echo "graphify not importable — attempting install..."
    INSTALL_LOG="graphify-out/.graphify_install.log"
    if ! "$PYTHON" -m pip install graphifyy -q 2>"$INSTALL_LOG"; then
        # PEP 668 environments (macOS system python, Debian 12+) need the flag
        if ! "$PYTHON" -m pip install graphifyy -q --break-system-packages 2>>"$INSTALL_LOG"; then
            echo ""
            echo "ERROR: could not install graphify. Last install output:"
            echo "---"
            tail -20 "$INSTALL_LOG"
            echo "---"
            echo ""
            echo "Common fixes:"
            echo "  • Network / corporate proxy: check your pip config"
            echo "  • Permissions: try 'pipx install graphifyy' instead"
            echo "  • Python version: graphify needs Python 3.10+"
            echo ""
            echo "Stopping. Do not re-run /graphify until the import works:"
            echo "  $PYTHON -c 'import graphify'"
            exit 2
        fi
    fi
    # Verify install actually took
    if ! "$PYTHON" -c "import graphify" 2>/dev/null; then
        echo "ERROR: graphify installed but still not importable by $PYTHON."
        echo "This usually means a venv / pyenv mismatch. Run:"
        echo "  which python3 && $PYTHON -c 'import sys; print(sys.path)'"
        exit 2
    fi
    echo "graphify installed successfully."
fi
 
# Write interpreter path for all subsequent steps
"$PYTHON" -c "import sys; open('graphify-out/.graphify_python', 'w').write(sys.executable)"

If the import succeeds on first try, print nothing and move straight to Step 2. If an install ran, the success line is the only output. If install failed, the skill has already exited — do not continue.

In every subsequent bash block, replace `python3` with `$(cat graphify-out/.graphify_python)` to use the correct interpreter.

Step 2 - Detect files

$(cat graphify-out/.graphify_python) -c "
import json
from graphify.detect import detect
from pathlib import Path
result = detect(Path('INPUT_PATH'))
print(json.dumps(result))
" > graphify-out/.graphify_detect.json

Replace INPUT_PATH with the actual path the user provided. Do NOT cat or print the JSON - read it silently and present a clean summary instead:

Corpus: X files · ~Y words
  code:     N files (.py .ts .go ...)
  docs:     N files (.md .txt ...)
  papers:   N files (.pdf ...)
  images:   N files

Then act on it:

• If total_files is 0: stop with "No supported files found in [path]."
• If skipped_sensitive is non-empty: mention file count skipped, not the file names.
• If total_words > 2,000,000 OR total_files > 200: show the warning and the top 5 subdirectories by file count, then ask which subfolder to run on. Wait for the user's answer before proceeding.
• Otherwise: proceed directly to Step 2.5 if the corpus has video files, else straight to Step 3 - no need to ask anything.

Step 2.5 - Transcribe video/audio files (only if detect reported video files)

Skip this step entirely if .graphify_detect.json's files.video list is empty. Video and audio cannot be read directly - transcribe first, then treat the transcripts as doc files in Step 3. See TRANSCRIBE.md for the full procedure (writing a domain-hint prompt, --whisper-model handling, running graphify.transcribe.transcribe_all).

Step 3 - Extract entities and relationships

Before starting: note whether --mode deep was given. You must pass DEEP_MODE=true to every subagent in Step B2 if it was. Track this from the original invocation - do not lose it.

This step has two parts: structural extraction (deterministic, free) and semantic extraction (Claude, costs tokens).

Run Part A (AST), Part A.5 (zero-LLM typed edges, if available), and Part B (semantic) in parallel. Dispatch all semantic subagents AND start the structural extractions in the same message. They operate on different surfaces — AST on code files, Part A.5 on markdown frontmatter + wikilinks, Part B on docs/papers/images via subagents — so they can run simultaneously. Merge results in Part C as before.

Note: Parallelizing structural + semantic saves 5-15s on large corpora. The structural passes are deterministic and fast; start them while subagents process the rest.

Part A - Structural extraction for code files

For any code files detected, run AST extraction in parallel with Part B subagents:

$(cat graphify-out/.graphify_python) -c "
import sys, json
from graphify.extract import collect_files, extract
from pathlib import Path
import json
 
code_files = []
detect = json.loads(Path('graphify-out/.graphify_detect.json').read_text())
for f in detect.get('files', {}).get('code', []):
    code_files.extend(collect_files(Path(f)) if Path(f).is_dir() else [Path(f)])
 
if code_files:
    result = extract(code_files)
    Path('graphify-out/.graphify_ast.json').write_text(json.dumps(result, indent=2))
    print(f'AST: {len(result[\"nodes\"])} nodes, {len(result[\"edges\"])} edges')
else:
    Path('graphify-out/.graphify_ast.json').write_text(json.dumps({'nodes':[],'edges':[],'input_tokens':0,'output_tokens':0}))
    print('No code files - skipping AST extraction')
"

Part A.5 - Zero-LLM typed-relationship extraction for markdown

If a typed-relationship extractor is available, run it in parallel with Part A and Part B. It walks markdown, reads frontmatter + body, and emits typed edges (works_at, floor_at, journaled_about, attended, mentions, governs, created_on, investor_for) to JSONL at zero LLM cost. Extracts ~80% of explicit edges (frontmatter + wikilinks); Part B's semantic extraction is configured to skip entities + edges already covered by this pass, so the LLM only fires on the genuinely-ambiguous ~20% (proper-noun disambiguation, novel entity types, negation parsing).