vinix24/vnx-orchestration/intelligence-engineer

Intelligence Engineer

VNX-specific domain expert for the intelligence system, central state, and dispatch lifecycle.

Core Mission

VNX has a sprawling but specific data model: dispatches flow through a runtime coordination DB, generate receipts and events, get archived, and feed back into a quality intelligence DB that informs future dispatch decisions. Most "intelligence not working" or "central state weird" bugs are NOT generic database bugs — they're domain bugs in VNX-specific code paths.

This skill exists alongside database-engineer (which covers SQLite mechanics generically). Use database-engineer for migration/schema/UPSERT work; use this skill when the question requires knowing which VNX table holds what, and why.

When to Use

• Editing or designing schemas in schemas/quality_intelligence.sql, schemas/runtime_coordination_v*.sql, or any 0010_… / 0015_… migration
• Touching scripts/quality_db_init.py, scripts/lib/coordination_db.py, or any module that initializes / queries central state
• Working with intelligence injection: scripts/lib/intelligence_*, scripts/build_t0_state.py, scripts/lib/intelligence_injection.py
• Debugging "code_snippets show wrong project", "dispatches missing from T0 state", "success_patterns not retrieved", "central DB query returns nothing"
• Querying central DB for cross-project insights (federation, aggregation, analytics)
• Designing a new intelligence-feeding mechanism (e.g. new event type → injection rule)
• Modifying any code that touches the project_id flow (registry → import → query → injection)

STEP 0 — Foundational Check (Mandatory)

BEFORE proposing any design, fix, or implementation:

1. Consult relevant ADRs in docs/governance/decisions/. Special attention to:

• ADR-005 (NDJSON audit ledger as primary observability)
• ADR-007 (multi-tenant project_id stamping; composite keys for central state DBs)
• ADR-010 (subprocess adapter as canonical Claude routing)

List any ADR that applies to the task and how it constrains your solution.

1. Consult project-local sources of past incidents: the ADRs above and the incident write-ups in claudedocs/. Do not read user-global Claude memory (~/.claude/projects/...) and never repeat memory content in a report or review.

1. Check P4-style incident docs in claudedocs/ for analogous failures (e.g., 2026-05-09-p4-migration-architecture-lessons.md for multi-tenant migration patterns).

1. State your foundational read aloud at the start of your response. Example: "ADR-007 applies: new tabel X needs composite PK over project_id. Per P4 §4.2, single-column UNIQUE is a smell."

Skipping STEP 0 is a process violation, not a shortcut. The FUT-1 chain (2026-05-28) burned 6 codex rounds because ADR-007 was not consulted at design time.

Decision Protocol

When asked a "where does X live in VNX?" question, FIRST consult references/quality-intelligence-schema.md and references/runtime-coordination-schema.md for table-by-table content. Don't guess; the schemas have evolved and the canonical answer is in the schema files + their accompanying quality_db_init.py imperative additions.

When asked to write code that produces or consumes intelligence, walk through references/dispatch-lifecycle.md first to understand which lifecycle stage produces the data you need.

1. The Two Central DBs

VNX has two physical SQLite databases that together hold the central state:

~/.vnx-data/state/quality_intelligence.db (QI)

Holds learned patterns, code snippets, and quality signals. Read primarily by intelligence injection at SessionStart and by analytics queries.

Key tables (full reference: references/quality-intelligence-schema.md):

• `code_snippets` (FTS5 vtab) — extracted code patterns with project_id column. Indexed for full-text search.
• `snippet_metadata` — companion table linking snippet_rowid → code_snippets.rowid. Holds pattern_hash, quality_score, source_commit_hash, usage_count.
• `success_patterns`, `antipatterns`, `prevention_rules` — learned governance rules with provenance.
• `pattern_usage` — cross-references of which dispatch used which pattern.
• `tag_combinations` — pre-computed tag tuples for fast filtering.
• `vnx_code_quality` — quality scoring per code area.

Schema source files: schemas/quality_intelligence.sql (base) + imperative additions in scripts/quality_db_init.py:84-...

~/.vnx-data/state/runtime_coordination.db (RC)

Holds dispatch lifecycle state. Read by dispatcher, T0 status, terminal coordination.

Key tables (full reference: references/runtime-coordination-schema.md):

• `dispatches` — every dispatch ever created. dispatch_id is the primary identifier across VNX.
• `dispatch_attempts` — retry attempts per dispatch.
• `terminal_leases` — which terminal owns which dispatch right now.
• `coordination_events` — append-only event log: state transitions, receipts, escalations.
• `incident_log` — flagged incidents (cooldowns, errors).
• `intelligence_injections` — record of which intelligence was injected where.
• `retry_budgets`, `retry_state`, `escalation_log` — retry/escalation bookkeeping.

Schema source: schemas/runtime_coordination.sql (v1 base) + delta migrations runtime_coordination_v2.sql through runtime_coordination_v10.sql (ls schemas/runtime_coordination_v*.sql for the current list). Apply order matters; bootstrap with coordination_db.init_schema(state_dir).

Why two DBs?

QI is mostly read (intelligence injection); RC is mostly written (dispatch flow). Separating them:

• Reduces lock contention (RC has lots of small writes, QI mostly batch reads)
• Isolates retention (QI snippets last forever, RC events archive after N days)
• Allows separate backup cadence

When you query "the central state", you usually need to ATTACH both:

con = sqlite3.connect(qi_path)
con.execute("ATTACH DATABASE ? AS rc", (f"file:{rc_path}?mode=ro",))
# Now can JOIN dispatches (rc.dispatches) with intelligence_injections

2. The Dispatch Lifecycle

The canonical flow that VNX governance is built on:

1. CREATE   T0 writes a dispatch file → .vnx-data/dispatches/pending/<id>/dispatch.json
2. PROMOTE  operator approval → moves to active/  → row in rc.dispatches (state='queued')
3. DELIVER  dispatcher delivers to terminal pane → state='claimed' then 'delivering', terminal_leases row leased
4. EXECUTE  worker runs; events streamed to .vnx-data/events/T<n>.ndjson
5. RECEIPT  worker writes receipt → t0_receipts.ndjson (with append_receipt() helper)
6. ARCHIVE  T0 reviews receipt → moves dispatch to archive/, terminal_leases released
7. INJECT   subsequent SessionStart pulls intelligence from rc.dispatches + qi.success_patterns into t0_state.json

Each step has an associated table or file. See references/dispatch-lifecycle.md for the full diagram + which step writes which row + retention rules.

Critical invariants:

• Every persistent record has project_id stamped at write time
• dispatch_id is unique within a project, NOT globally — use prefix-rewrite (<project_id>:<dispatch_id>) for cross-project federation
• Receipts are NDJSON (one row per receipt), atomically appended via fcntl.flock
• Events files are ring-buffered: T<n>.ndjson is the live file (truncated post-dispatch), durable record in events/archive/<terminal>/<dispatch_id>.ndjson

3. Intelligence Injection Flow

Intelligence injection is how VNX "learns": insights from past dispatches get pulled into the current T0 session. Without this, T0 starts every conversation cold.

Flow:

SessionStart hook → build_t0_state.py
  ├─ Reads central QI (success_patterns, antipatterns) for this project_id
  ├─ Reads central RC (recent dispatches, open items, escalations)
  ├─ Filters by scope match (project_id, recency, relevance)
  └─ Writes t0_state.json with `strategic_state` + `intelligence` blocks
T0 reads t0_state.json on every SessionStart automatically

Code paths:

• scripts/build_t0_state.py — main projection driver
• scripts/lib/intelligence_injection.py — scope match + filter logic
• scripts/aggregator/build_central_view.py — cross-project queries (federation aggregator, P1 work)

Key concept: scope match. An injected pattern must be relevant to the current dispatch. Scope keys include: project_id, track, tags, risk_class. The match logic re-evaluates after canonical project_id remap (per feedback_dispatcher_role_alias memory).

Common bug: intelligence injection writes to runtime_coordination.intelligence_injections table but the scope match was wrong → injected pattern was irrelevant → operator had to manually filter. Fix: tighten scope match logic, add tests for edge cases.

4. Schema Evolution

VNX schemas have evolved through 9+ versions for RC and 16+ migrations for QI. Key checkpoints:

• `schemas/runtime_coordination.sql` is v1. Each runtime_coordination_v<N>.sql is a DELTA, not a full schema. Apply v1 → v2 → … → v10 in order (init_schema does, then runs the v11 index step).
• `schemas/quality_intelligence.sql` is the base. scripts/quality_db_init.py:84- adds tables imperatively (e.g. confidence_events, dispatch_pattern_offered). Don't bypass bootstrap_qi_db() — it knows the full set.
• `schemas/migrations/0010_add_project_id.sql` — adds project_id to "hot" RC tables (dispatches, dispatch_attempts, terminal_leases, etc).
• `schemas/migrations/0015_complete_project_id.sql` — extends to remaining tables (vnx_code_quality, snippet_metadata, etc).
• `schemas/migrations/0016_rebuild_fts5.sql` — rebuilds code_snippets FTS5 vtab to include project_id column.

For new migrations: number sequentially, document in the file header what it does and why, include CREATE INDEX for any JOIN columns. See database-engineer/SKILL.md for general migration patterns.

5. The project_id Flow

Where project_id comes from and how it propagates:

1. Registry: ~/.vnx/projects.json — operator-curated list of projects. Each entry has name and path.
2. Synthesis: scripts/aggregator/build_central_view.py:synthesize_project_id(name) slugifies the name into a project_id token (lowercase, alphanumeric+dash, max 32 chars).
3. Per-project state: each project's .vnx-data/state/{quality_intelligence,runtime_coordination}.db — rows MAY have project_id column (legacy vnx-dev default for autopilot's pre-existing rows; absent for newer source DBs).
4. Migration import: P4 migration script reads source rows, OVERRIDES any source project_id value with the project's actual id from registry. Stamping happens in _import_table line ~1010.
5. Central DB: every row has project_id set to the registry-derived value. Cross-project queries always filter or aggregate by this column.
6. Intelligence injection: scope match uses project_id. Cross-project federation (P1) uses project_id as discriminator.

Pitfall: if ~/.vnx/projects.json has a project under name vnx-roadmap-autopilot and migration imports it, the central rows are stamped vnx-roadmap-autopilot. Renaming the entry to vnx-orchestration later doesn't update existing central rows — they keep the old slug. P4 round-5 caught a flavor of this. Test rename scenarios explicitly.

6. VNX Intelligence Cookbook

Common queries you'll need:

List all dispatches per project (last 7 days)

ATTACH DATABASE ? AS rc;
SELECT project_id, COUNT(*) AS dispatches, MAX(created_at) AS most_recent
FROM rc.dispatches
WHERE created_at > datetime('now', '-7 days')
GROUP BY project_id
ORDER BY dispatches DESC;