Skill v1.0.0
currentAutomated scan100/100name: sightmap-browser description: Use when driving a live browser session with a sightmap — starting or navigating a Chrome session, taking annotated component snapshots to read page state, and interacting with elements by component ID or component query. Activate when a .sightmap/ directory is present and you need to observe or act on the running app. activation:
- .sightmap/ directory exists in project
- driving a live browser session against a sightmap corpus
metadata: version: "1.0.0"
Sightmap browser: observe and interact
The sightmap CLI drives a Chrome-for-Testing session and layers your .sightmap/ corpus onto the live page, so you can read page state as an annotated component tree and act on elements by their semantic component identity instead of brittle CSS. To build or maintain the corpus itself, see the sightmap-authoring skill.
What to reach for:
- `snapshot` — read page state as an annotated component tree (start here).
- interaction (
click/fill/scroll/…) — drive the page by component identity. - `browser screenshot` — visual evidence; clip to one component with
--component. - `console` / `network` — debug what the page logged and requested.
- `inspect` — raw DOM for authoring selectors (see
sightmap-authoring).
Installation
Every command below needs the sightmap binary on your PATH. Check first, install only if missing — run sightmap version; if that fails, install it:
npm install -g @sightmap/sightmap # recommended: global installsightmap version # verify it's on PATHgo install github.com/sightmap/sightmap/go/cmd/sightmap@latest # or build from source
This skill drives a multi-step browser session, so prefer a global install over npx @sightmap/sightmap <command> (which re-resolves the package on every invocation). Use npm in instructions — the published package is identical under pnpm/yarn, but npm is the safe baseline.
Session management
browser start is a long-running foreground daemon: it owns Chrome and the console/network collector for the whole session, so it holds the shell until you stop it. Every other command here is a thin client that connects to that daemon and returns immediately.
- Interactive: run
startin its own terminal, drive from another. - Scripts / agents (shells that run each command to completion): use
`sightmap browser start --detach`. It backgrounds the daemon in its own session, waits until it is serving, and returns — so it never hangs your shell, and (unlike nohup start &) the daemon survives between one-shot commands.
- A client command that hangs almost always means the daemon isn't up —
run browser status, don't wait it out.
- On a headless Linux host
startauto-detects the missing display and runs
headless; if Chrome hits a sandbox restriction the error tells you to add --chrome-flag=--no-sandbox.
| Command | What it does | |
|---|---|---|
sightmap browser start | Launch Chrome + the overlay server. Foreground daemon — holds the shell; pass `--detach` in scripts/agents. Creates .sightmap/ if absent and writes .sightmap/.session (fails loudly if it can't). Every other command finds the session via this file, so with no session file a command warns and falls back to the default CDP port — which may be a different agent's Chrome; pass --addr to be explicit. | |
sightmap browser status | Check session health, tabs, and current URL. Reports ⚠ degraded when Chrome's CDP is up but the daemon's HTTP server was reaped. | |
sightmap browser navigate 'URL' | Navigate to URL (positional arg — no --url flag). | |
sightmap browser stop | Stop Chrome session. | |
sightmap browser eval 'js' | Evaluate JS in page context. Returns JSON-serializable values only — DOM element references return an error. A returned promise is awaited, so async evals resolve to their value. | |
sightmap browser inject --file X --persist | Inject a script that re-runs on every new document/tab for the whole session (survives navigations), vs eval's one-shot. --list / --remove ID manage them. Needs a running session (the daemon holds the registry). | |
sightmap browser mcp list | Enumerate the WebMCP tools the page exposes via document.modelContext (name, description, input schema; --json for full schemas). Reports native vs polyfilled vs absent; fails loudly with the Chrome-flag hint when absent. When a page exposes tools, prefer a named mcp call over blind click/fill. | |
sightmap browser mcp call <tool> --args '{…}' | Invoke one exposed tool via executeTool. Args via --args JSON and/or repeatable --param k=v. Unwraps the WebMCP CallToolResult (text/guidance shown as itself, not a stringified blob) and exits non-zero on tool isError. One tool at a point in time — no cross-navigation; if the tool navigates, re-run mcp list on the new view. | |
sightmap browser screenshot --out FILE.png | Screenshot the page. Clip to a component with --component NAME (or --selector CSS), optionally --expand-pct N for context. |
Reading the page: annotated snapshots
sightmap snapshot --url URL prints the page's component tree with your corpus layered on (add --coverage for a terse coverage-only view that suppresses the tree). Each line starts with a numeric component ID, then the node content:
42 [SiteHeader]83 [DesktopNavLink label="Explore"]91 link "Privacy Policy"
- `[ComponentName prop="val"]` — sightmap-matched node; role suppressed
- `role "text"` — unmatched node; role and accessible name shown
- `42` prefix — probe component ID; usable as a handle for interaction commands
Properties are sorted alphabetically. The accessible name is shown last inside brackets when not suppressed (an exact match with a property value).
If the page has 0 interactive nodes, coverage renders ∅ (not ✓) and snapshot exits non-zero — the page is blank or still loading. Wait for it with browser wait-for --selector ... (or --wait N) and re-snap before acting.
Interaction (by ID or component query)
click, fill, hover, and scroll --component-id accept either a numeric probe ID from snapshot output or a component query (see below):
| Command | What it does | |
|---|---|---|
sightmap browser click <id> | Click element by probe ID | |
sightmap browser click 'ComponentQuery' | Click element by sightmap identity (preferred on dynamic pages) | |
sightmap browser fill [--clear] <id-or-query> "text" | Type into an input. Errors if the value doesn't stick — retry with --clear (see gotchas). | |
sightmap browser hover <id-or-query> | Hover over element | |
sightmap browser keypress Enter | Press a key (optionally focused on <id>) | |
sightmap browser scroll --delta-y 500 | Scroll the page | |
sightmap browser scroll --component-id <id-or-query> | Scroll a component into view | |
sightmap browser click --x N --y N | Click raw coordinates (escape hatch; layout-fragile) | |
sightmap browser wait-for --view <Name> | Wait until the URL resolves to a sightmap view (the step boundary after a navigating action) | |
sightmap browser wait-for --component '<Query>' | Wait until a component query matches a node | |
sightmap browser wait-for --selector "[data-loaded]" | Wait until a CSS selector matches (also --url SUBSTRING (plain substring, not glob), --load) | |
sightmap browser tabs list | List open tabs |
Component queries (CSS-shaped, over sightmap components + extracted properties)
Probe IDs are extraction-local: they're reassigned every snapshot, so on re-rendering pages an <id> from an earlier call goes stale. A component query re-resolves against the live tree atomically (extract → match → act in one pass), so nothing goes stale. Prefer queries on dynamic pages.
| Query | Resolves to | |
|---|---|---|
FulfillmentTileButton[label=deliveryTile] | the FulfillmentTileButton whose label property equals deliveryTile | |
ProductCard[name^="Webber grill"] | a ProductCard whose name starts with Webber grill | |
ProductCard[name*="weber" i] | substring, case-insensitive (i flag) | |
LoginForm UserNameInput | a UserNameInput anywhere under a LoginForm (target = last component) | |
FulfillmentTileButton#1 | occurrence 1 (0-based) when several match — weak fallback |
- Brackets are component names;
[prop op value]filters on **extracted
properties** (not raw DOM attributes). Operators: = exact, ^= prefix, *= substring, with an optional trailing i for case-insensitive.
- Whitespace is a descendant combinator; the last component is the target.
It matches at any depth (A B = a B anywhere under an A), and a predicate on an ancestor scopes the target (Row[title="…"] Star). There is no `>` child combinator — A > B is a parse error; use whitespace.
- If a query matches zero components it errors; if it matches several it
errors and prints the candidates with their distinguishing properties — add a predicate to disambiguate, or append #N. ("Several match" is an ambiguity error listing candidates, not a "no such component" error — read it and disambiguate rather than assuming the component is unreachable.)
- Robust queries need a corpus with disambiguating properties; authoring a
property like FulfillmentTileButton.label is what makes the element addressable (see the sightmap-authoring skill).
--sightmap-dir(default.sightmap) controls which corpus resolves a query.
Console & network — the runtime view of the corpus
The browser start daemon owns the session and runs a collector that buffers console messages and network requests from every tab for the life of the session (bounded ring buffers, ~1000 of each). Read them with thin query commands — no re-attaching to Chrome, and with history the transient page commands never saw.
These are not just raw debug streams. Each captured record is matched against the corpus's requests: and messages: entries (see the sightmap-authoring skill), so every line leads with a `[MatchedName]` slot — or [--] when nothing in the corpus classified it — the way a snapshot foregrounds a node's [Component]. Matched records also trail their extracted `{name=value}` properties.
| Command | What it does | |
|---|---|---|
sightmap console list [--level error] [--tab ID] [--limit N] | Console messages, each led by its messages: match (uncaught exceptions fold in as level=exception). | |
sightmap console get <index> | One console message + any extracted stack properties:. | |
sightmap network list [--type XHR] [--url /api] [--tab ID] [--limit N] | Requests: [Match] method url → status (type) {prop=value}. | |
sightmap network get <index> [--response-file F] | One request + its requests: match, a Properties: block, and response body (save with --response-file). |
Reading the slot:
[AuraAction] POST /aura → 200 (XHR) {first_action_state=SUCCESS} # matched a requests: entry + extracted a body value[AuraAction] POST /aura → 200 (XHR) {first_action_state=ERROR} # 200 OK, but the body says it failed — the payoff of request properties:[--] GET /favicon.ico → 200 (Image) # unmatched: no corpus entry, unstructured[DeprecatedChartApi] warn ...stackhbar.js has been deprecated # matched a messages: entry
The [--] slot is deliberately quiet — it marks a record unclassified without drowning the matched ones.
- These need a running
browser startsession — that's where the collector
lives. Records from before the session started aren't captured, so in an attached/degraded session (or right after start) reproduce the traffic: refresh the page or re-trigger the action, then read. (A SPA may never reach wait-for --load; prefer wait-for --url/--selector, or just re-read after a short pause.)
--urlis a substring match, not a path anchor (--url /auraalso catches
.../auraFW/...) — narrow it when a route is noisy.
- To find failures: reproduce, then scan the
[...]slots inconsole list
(--level error) and network list; network get <index> pulls the body + properties.
Gotchas
`browser navigate` takes a positional URL, not a `--url` flag.
sightmap browser navigate 'https://...' # ✓ correctsightmap browser navigate --url 'https://' # ✗ wrong — passes literal "--url" as the URL
`browser fill` detects when a value doesn't stick. On some React-controlled inputs plain fill (select-all + type) leaves the field empty. fill now reads the value back and errors in that case, telling you to retry with --clear (which clears via the native value setter first). If filling the same field repeatedly accumulates text, also use --clear.
sightmap browser fill --clear 'SearchInput' 'burrito'
`browser click` scrolls the target into view and refuses off-screen / covered clicks. click scrolls the element to the center of the viewport, then verifies its center is actually the top-most element there before dispatching. It errors (rather than silently no-op'ing) when the target can't be positioned in the viewport, or is covered by another element — the signal of an open overlay/modal or a target you need to reveal first. The confirmation line reports the real post-scroll coordinates it clicked.
After an action that should navigate, wait for the destination — don't snapshot immediately. SPA (client-side) navigation lands after click returns, so a snapshot taken right away shows the old page. click deliberately does not guess or wait; make the wait an explicit step, the way Playwright/Selenium do:
sightmap browser click 'WorkItemRow[key="FALCON-7"]'sightmap browser wait-for --view WorkItemDetail # or --component 'WorkItemDetail', or --url '/browse/'
wait-for auto-retries until the postcondition holds or it times out (loudly). Record per-app async quirks (which actions navigate, slow routes) in the relevant view/component memory so the next agent knows to wait.
`browser eval` cannot return DOM elements. Only JSON-serializable values are returned. document.querySelector(...) returns an error reference. Extract what you need instead: document.querySelector("sel")?.textContent.
Stale session files. browser status probes the CDP endpoint (not just the .session file): it reports ✗ unreachable and removes the stale session file when Chrome is gone. If a command still fails with a CDP error, run browser start again.
Always pass `--tab` when several tabs are open. Page commands auto-pick the lone content tab but error (listing tabs) when zero or several are open. browser start prints your tab ID; thread it through as --tab <ID>.
`browser navigate` prints the final URL. After a redirect — server-side or client-side (an SPA auth guard bouncing /login → /, or / → /workspace) — navigate prints (redirected to FINAL) so you know where you actually landed, not just the URL you asked for.