owwzo/ai-agent/opencli-browser

opencli-browser

The first reader of this CLI is an agent, not a human. Every subcommand returns a structured envelope that tells you exactly what matched, how confident the match is, and what to do if it didn't. Lean on those envelopes — do not guess.

This skill is for driving a live browser to accomplish an agent task. If you are building a reusable adapter under ~/.opencli/clis/<site>/ use opencli-adapter-author instead.

Orchestration and low-cost page inspection

For browser tasks, use code_mode as the default orchestration layer. Batch independent browser commands in a single code_mode script via browser.command({args:[...]}); this is the browser equivalent of composing CLI commands in one script and reduces model/tool round trips. Do not call a site's OpenCLI adapter when the user requests direct browser operation or explicitly forbids that adapter.

Use a probe-then-batch workflow:

1. Inspect the page once to learn the relevant structure and selectors. Prefer a narrow find, targeted get html --selector ... with strict depth/text limits, or a read-only browser eval that returns only the specific DOM fields needed. Use full state only when refs or broad structure are necessary.
2. Once selectors, page structure, and workflow are clear, write one code_mode JavaScript script to batch extraction and repetitive work. Keep the script's outputs compact: return only requested fields, counts, and concise status/error details.
3. Avoid re-running full state on the same unchanged page or dumping its entire output repeatedly. After a page transition, reacquire only the minimum fresh snapshot/locator information needed because refs may have changed; otherwise use targeted reads and verify postconditions.
4. Keep page-context JavaScript strictly read-only: use browser eval for targeted DOM/data inspection only. Do not use page JS to click, scroll, navigate, submit, type, or otherwise mutate the page. Use the structured browser primitives (click, scroll, type, fill, keys, etc.) for all interactions; orchestrate those primitives through code_mode when batching is useful.
5. To reduce site risk controls, do not simulate interaction with JavaScript or fire synthetic DOM events. Use the browser primitives for user-like interactions, keep the action rate reasonable, and stop if the site presents a challenge or blocks access.

Prerequisites

opencli doctor

Until doctor is green, nothing else will work. Typical failures: Chrome not running, extension not installed, debug port blocked by 1Password / other extensions. The doctor output tells you which.

Session lifecycle

• opencli browser * commands require a <session> positional immediately after browser. Use the same session name for a multi-step flow; use a different name to isolate parallel browser work.
• Use a stable session name for any multi-command or human-paced browser workflow. Example: opencli browser fb-yaya-warmup open https://example.com, then reuse opencli browser fb-yaya-warmup state, extract, click, etc.
• Owned Browser Bridge sessions keep a tab lease alive between calls; the extension lease is the only owner of the session's default tab. The CLI does not cache a second default target. Release it with opencli browser <session> close or let the idle timeout expire.
• opencli browser <session> bind binds the Chrome tab you already have open to that session. Use this for logged-in pages, SSO flows, or pages you manually positioned before handing control to the agent.
• --window foreground|background (or OPENCLI_WINDOW=foreground|background) chooses whether OpenCLI creates/focuses a foreground browser window or uses a background browser window for owned sessions.

Bind Tab

opencli browser gmail bind
opencli browser gmail state
opencli browser gmail click "Search"
opencli browser gmail network
opencli browser gmail unbind

Binding never owns the user window and never closes the user tab. It fails closed if the tab is closed or becomes non-debuggable. Re-run opencli browser <session> bind when you switch to a different real tab.

Navigation is allowed on bound sessions because the session now represents explicit agent ownership of that tab. Tab mutation (tab new, tab select, tab close) is still blocked for bound sessions. Use an owned session when you want OpenCLI to manage tab lifecycle.

Bound sessions have no OpenCLI idle-close timer; the binding lasts until unbind, tab close, window close, or daemon restart.

Mental model

1. Selector-first target contract. Every interaction command (click, type, select, get text/value/attributes) takes one <target>, which is either a numeric ref from state/find or a CSS selector. Use --nth <n> to disambiguate multiple CSS matches.
2. Every envelope reports `matches_n` and `match_level`. match_level is exact, stable, or reidentified — the CLI already rescued moderate DOM drift for you, but the level tells you how confident to be.
3. Compact output first, full payload on demand. state is a budget-aware snapshot; get html --as json supports --depth/--children-max/--text-max; network returns shape previews and you re-fetch a single body with --detail <key>. If you emit a giant payload you are burning context you did not need to burn.
4. Structured errors are machine-readable. On failure the CLI emits {error: {code, message, hint?, candidates?}}. Branch on code, not on message strings.

Critical rules

1. Always inspect before you act. Run state or find first. Never hard-code a ref or selector from memory across sessions — indices are per-snapshot.
2. Prefer site adapters before raw browser driving. If opencli <site> <command> already covers the task, use that adapter command first (opencli facebook notifications, opencli reddit read, opencli chatgpt model <level>, etc.). Use opencli browser ... only for gaps, debugging, or one-off UI flows the adapter does not expose.
3. Prefer numeric ref over CSS once you have it. Numeric refs survive mild DOM shifts because the CLI fingerprints each tagged element. A CSS selector written by hand will break the first time the site re-renders.
4. Read `match_level` after every write. exact = all good. stable = the element is the same but some soft attrs drifted — your action still applied. reidentified = the original ref was gone and the CLI found a unique replacement; double-check you hit the right element.
5. Use the `compound` field for form controls. Do not regex-guess a date format, do not state twice to get the full <select> options list. The compound envelope has the format string, full option list up to 50, options_total for overflow, and accept/multiple for <input type=file>.
6. Verify writes that matter. After type <target> <text>, run get value <target>. After select, run get value. Autocomplete widgets, React controlled inputs, and masked fields all silently eat characters. The CLI cannot detect this for you.
7. `state` → action → `state` after a page change. Navigations, form submits, and SPA route changes invalidate refs. Take a fresh snapshot. Do not reuse refs from before the transition.
8. Chain with `&&` when reusing freshly parsed refs. A chained sequence runs in one shell so the ref you just read from output can be passed directly to the next command. Separate shell invocations keep the named browser session, but any shell-local variables or copied refs from the previous command can go stale after page changes.
9. `eval` is read-only. Wrap the JS in an IIFE and return JSON. If you need to change the page, use the structured click / type / select / keys commands instead — they produce structured output and fingerprints, eval does not.
10. Prefer `network` to screen-scraping. If a page you care about fetches its data from a JSON API, the API is almost always more reliable than scraping the rendered DOM. Capture once, inspect the shape, then --detail <key> the body you need.

Download output isolation

Download adapters write files into the current task's workspace. Always pass a task-specific output directory to every download command, and use a different directory for parallel downloads. Do not reuse a shared download directory.

const output = "./xiaohongshu-agent-post-images";
return await browser.command({
  args: ["xiaohongshu", "download", "<signed-note-url>", "--output", output],
});

After a download completes, use the returned artifact/file event for file URLs. Do not read or expose server absolute paths. The Java tool layer also assigns a unique output directory when an older download call omits --output, but new calls must declare it explicitly.

Sitemaps

If browser open or browser analyze returns sitemap.available: true, switch to opencli-browser-sitemap before continuing a multi-step site flow. The sitemap is prior context for pages, actions, workflows, APIs, and pitfalls; it is not truth. If the browser state disagrees with the sitemap, trust the browser and mark the sitemap stale via opencli-sitemap-author.

Target contract (<target> for click / type / select / get text|value|attributes)

<target> ::= <numeric-ref> | <css-selector>

• Numeric ref — the [N] index from state or find. Cheap, resilient to soft DOM drift.
• CSS selector — anything querySelectorAll accepts. Must be unambiguous on write ops, or pair with --nth <n>.

Envelope on success

{ "clicked": true, "target": "3", "matches_n": 1, "match_level": "exact" }

{ "value": "kalevin@example.com", "matches_n": 1, "match_level": "stable" }

match_level

Structured error codes

Branch on these, not on the human message:

Error envelope always includes error.code and error.message. Target errors (selector_not_found, selector_ambiguous, etc.) often add error.candidates: string[] with suggested selectors. option_not_found adds error.available: string[] instead.

Command reference

Inspect