buriedsignals/spotlight/integrations

What's the task?
│
├── "Navigate a form / click through a UI / extract from a JS-rendered page"
│     → dev-browser — NATIVE harness capability (interactive / authenticated / JS-rendered /
│       chain-of-custody capture); available to ALL tiers, not an integration, not preflight-gated
│     → for STATIC content extraction, use the Crawl4AI scraping seam (also native) — a different
│       job, not a dev-browser fallback (the two serve different purposes)
│     → legacy alternates only if dev-browser is genuinely unavailable: browse (curated browse.sh),
│       then browser-harness / browser-use; last resort fetch() static scrape (weak on JS-heavy pages)
│
├── "Find deleted social posts / track narrative spread / cross-platform search"
│     → junkipedia  (if green — check preflight)
│     → fallback: search() + social-media-intelligence skill (limited without Junkipedia's archive)
│
├── "Analyse a whole social corpus — entity stance, narrative themes, who is amplifying,
│    engagement over time" / "cite a social post that may be deleted at origin"
│     → arbiter  (if green — check preflight) through the native HTTPS API; browse
│       the topic menu for a case study that already covers the event, or create
│       one from a reviewed plan when nothing matches
│     → requires the member's own `ARBITER_API_KEY`; `/arbiter` provides the
│       attributed signup link and local key setup guidance
│     → post pulls and agent questions are credit-metered to that member's Arbiter account
│     → fallback: junkipedia, then search() + social-media-intelligence skill
│
├── "Collect a bounded set of public posts from an X / Instagram / TikTok / Facebook /
│    LinkedIn profile, thread, or keyword search"
│     → native platform API first (YouTube Data API, Bluesky AT Protocol, Reddit API, Telegram —
│       see social-media-intelligence "Platform Tools")
│     → apify  (if green — check preflight) via `integrations/apify/integration.md`; write the
│       actor input to a case-local JSON file, cap `maxItems`, record collection authority
│     → fallback: fetch(profile_url) + manual review; junkipedia for archived/deleted posts
│
├── "Find accounts from one or more usernames / handles / aliases"
│     → maigret if preflight is green and the operator accepts account-discovery noise
│     → output is unverified account-discovery leads only
│     → fallback: search() + social-media-intelligence skill
│
├── "What runs this website / who registered the domain / which subdomains, trackers,
│    certificates or past versions does it have"
│     → invoke-skill("technical-investigation") first: it decides whether the target is a
│       suspicious indicator and records each path as observed / null / skipped / blocked
│     → webcheck  (if green — check preflight, sensitive mode false) via
│       `integrations/webcheck/integration.md`; lookup checks by default
│     → page checks (`contact_target: true`) contact the live host: only with the user's
│       explicit approval and an OPSEC plan in the methodology, never for a suspicious indicator
│     → fallback: crt.sh / RDAP / Wayback by hand as described in infrastructure-history
│
├── "What does this app / capture / assembly contain: endpoints, trackers, IPC, permissions"
│     → rea  (if green — check preflight) via `integrations/rea/integration.md`; the artifact
│       must already be in the case directory
│     → static only: never run the target; record how the artifact was obtained
│
├── "Methodology tool selection" / "Need a tool I don't know for category X" / "Compare tools" / "Niche country-specific tool"
│     → osint-navigator  (first pass ONLY if entitled + green + sensitive mode false — subscription tier)
│     → otherwise (DEFAULT for local / open tier): execute-shell("scripts/spotlight-python scripts/osint-tools.py find \"<keywords>\" [--category X]")  — local SQL index, 12,500 tools, offline, no Navigator needed
│
├── "Static page scrape / web search"
│     → fetch / search (verbs, no integration needed)
│
├── "Keep watching this after the cycle ends"
│     → invoke-skill("monitoring") to present a recommendation and, with
│       explicit approval, hand it to Mycroft. Spotlight never calls Scoutpost.
│
├── "Find a legal open-access copy of an academic paper with a DOI"
│     → unpaywall  (if green — check preflight)
│     → fallback: invoke-skill("content-access") and continue with CORE / Semantic Scholar
│
├── "Chain-of-custody evidence capture (court records, gov portals)"
│     → dev-browser + web-archiving, recorded in evidence-bundle.json
│
└── "Paywalled / gated content"
      → invoke-skill("content-access")  (hierarchy before marking inaccessible)

execute-shell("scripts/spotlight-python integrations/preflight.py --model-tier {config.model_tier} --json")