sirallap/agentglass/browser-use

Using the built-in browser

curl gets you the signed-out version of everything that matters, because the session lives in a browser. agentglass has one, already signed in to whatever the person using it is signed in to. agentglass-browser drives it, and it is built for you rather than lent to you: the whole DevTools protocol is here, so is running JavaScript, so is faking a broken API.

Start here, in this order

agentglass-browser health                     # is anything listening; always answers
agentglass-browser open https://example.com/app
agentglass-browser observe --shot             # EVERYTHING at once

observe is the verb to reach for first. One answer with the url and title, whether the view is visible and focused, the console and the network since last time, a tree of the interactive page addressed by role and accessible name, the current value of every input, and optionally the picture. Polling six verbs in turn is where the time goes.

After the first look, ask for only what changed:

agentglass-browser observe --delta            # {delta:true, added, removed, changed, same, console, network}
agentglass-browser click e12 --observe        # `after` is a delta too

added are whole nodes, removed are ids gone from the page, unlisted are ids still there but past the tree's cap, changed is {e, field: new value} (null = the field went away), same is how many did not move. New console and network rows only. form, storage and viewport appear only when they changed — absent means unchanged. Positions (at) are not diffed. After a navigation, with no earlier look, or after a look you only saw part of (--max-tokens, --summary), you get the full answer with delta:false and a reason. Plain observe is always the full page.

A page that is mostly below the fold or a menu you already know the id of does not need the whole tree:

agentglass-browser observe --viewport         # only what is on screen; `outOfView` counts the rest
agentglass-browser observe --root e12         # only the subtree of e12 (refused if e12 is gone)

Both are looks at a part: they are not a baseline, so they neither replace nor break the one --delta is measured against, and they cannot be combined with it. Unnamed layout wrappers (role presentation/none/generic with no name, id or test id) are left out of every tree, counted in wrappers, and an empty name is not written.

The baseline (and checkup's "since your last checkup") is kept per --as name; callers without one share a single baseline, so pass --as to get your own.

click and press wait for what they caused (the navigation, or a quiet page with no request in flight, capped at 1 s) and answer with an effect: navigated, newDocument, newErrors, failedRequests, dialog, settledBy. Often that is all you need to know — no look at all.

Did it break? One call

The edit → reload → "is it broken?" loop is one verb:

agentglass-browser checkup http://localhost:5173/   # load it, wait for quiet, report
agentglass-browser checkup --reload                 # after your edit
agentglass-browser checkup                          # no navigation: since your last checkup

The first field is the verdict, ok or N problems. Problems are uncaught exceptions and console errors — including the ones thrown while the page loaded, which console and observe cannot see — failed requests (4xx/5xx, CORS, blocked) and visible error text (role=alert, so an alert toast counts). Chromium's issues, perf (LCP, CLS) and a11y (unlabelled controls, with ids) come along as advice and do not count. A screenshot path only when something failed (--no-shot to skip; shot: "unavailable: …" when there is no frame to take).

Then the whole interaction in ONE call

agentglass-browser do "click #save" "wait #done" --observe

Starting this process costs ~69 ms before it says a word, so six separate verbs spend most of a second on startup alone. do spends it once. Measured: 618 ms → 94 ms for six verbs. Steps stop at the first failure, which comes back with its own console errors and failed requests.

Every round you spend re-reads everything above it, so the count of calls is the cost. Three rules:

1. Act, wait for the end state and look, in ONE call. A form is one

fill --field e3=... --field e4=... --submit --observe; a click that should lead somewhere is do "click e9" "wait #done" --observe. Do not split an action from the look that checks it.

1. Trust the receipt. click and press already say whether the page

navigated, errored, opened a dialog or opened a tab (effect.popups: [{page, url}], so the next call is --page <page>), and --observe already shows the page after. fill names any field the page changed from what you typed (changedByPage); no entry means every value took. Do not observe again to confirm what the answer just told you; look again only when it says something you did not expect. An act waits up to 3 s for its selector to appear and for a passing overlay to clear, so do not wait before a click on something that is about to render. An id the page re-rendered is refused with the id that replaced it (... is now e43): repeat the call with that one.

1. Stop when the result is on the page. The task is done the moment the

page shows it (a confirmation, the new URL, the saved value). Do not check it a second way, take a screenshot "to be sure", or re-read the page.

A loop or a condition is ONE script call

A task with a loop or a condition is one script call, not a round per verb. CODE is Python (a quoted string, or - for a heredoc). Every verb is a function that returns its value or raises BrowserError with the server's sentence; print() and a variable named result come back in one JSON answer with the page after (a delta when you observed it before; --no-observe drops it).

agentglass-browser script --as review - <<'PY'
for n in range(20):
    rows = extract(fields={"names": ".orders td.name"})
    if "Acme Ltd" in str(rows):
        break
    click("#next")
else:
    raise BrowserError("Acme Ltd is not on the first 20 pages")
click("tr.row-acme button.open")
result = {"opened_on_page": n + 1}
PY

Verbs: open observe click type press select wait waitfor text read links extract eval, fill({sel: value}, submit=True), and b("op", **args) for any other. A failure stops the script with line N: <sentence>; the printed output and the final observe still come back. The ceiling is 200 verb calls and 120 s. The code runs with the CLI's own rights, the same your shell already has. script is CLI-only: the MCP tools have no equivalent.

Wait for something instead of polling for it

agentglass-browser events --wait 30           # answers the MOMENT something happens

The cost of polling is not the clock, it is the twenty answers sitting in your context for the rest of the session. events waits on the server and answers once. "Nothing happened in thirty seconds" is an answer, not a failure.

The verbs, by what you reach for them for

dev loop    checkup (did it break — errors from load on, failed requests, visible errors)
measure     vitals (LCP/CLS/INP/TTFB/FCP, rated) · a11y (unlabelled controls, alt, heading jumps, lang)
look        observe · read · markdown · text · html (--clean: scripts/styles out, eN ids in) ·
            region · shot · frames · console · network · extract · links · count · search
            interactive · forms · attr
page        resize · zoom (the one Ctrl+/Ctrl- move) · emulate · throttle
handoff     handoff "why" [--until sel|/path] — the person does the CAPTCHA/2FA/consent, you continue
act         click · type (also rich editors: contenteditable) · select · check · fill · hover · dblclick · rightclick
            focus · blur · press · scroll · drag · upload · dialog (answer the next confirm/prompt)
wait        wait · waitfor (--until network-idle | no-timers) · events
many pages  scrape URL... (--read markdown|links|extract… --concurrency 1-4): a tab each, closed after
navigate    open · back · forward · reload
finished    done — when your task in the browser is over, run `agentglass-browser done` (MCP: browser_done): the person's overview shows Done instead of guessing from silence. Touches no page, allowed read-only and while the person has the tab; your next verb marks the tab working again.
tabs        tabs · tab · newtab · closetab · profiles (open|newtab --wait-slot S queues only when all 12 awake are in front or held; otherwise the 13th page puts the least-used one to sleep)
containers  whoami · profiles (--make/--drop) · newtab --profile · lanes
identity    cookies · storage · session save/load (MCP: storage_state) · permission · permissions · clipboard
templates   template save/list/rm NAME (CLI-only, no MCP tool) — a named session for `lane new --from-template`
run code    eval · eval --file · addInitScript · expose · exposed
page tools  tools · call-tool NAME --args '{"k":"v"}'   (only with AGENTGLASS_BROWSER_WEBMCP=1 on the server)
inspect     cdp · debug · listeners · coverage · trace · screencast (start · frames · stop · watch --out DIR)
devtools    inspect open|close · inspect panel <id> · inspect zoom <n> · inspect shot
network     fake · intercept · throttle · headers · har
pretend     emulate · resize · clock · settings
evidence    shot · shot --marks (eN labels on the picture) · shot --with-inspector · record · pdf · save · download · audit --script
batch       do (and `lanes` for several pages at once)

The structured readers — what an agent actually wants from a page

read gives you the page, but as one wall of text. When the question is a question, not "give me the page", reach for the verb that answers it:

agentglass-browser markdown                    # the page as markdown — headings, lists, code, links
agentglass-browser extract --field price=.price --field title=h1   # named fields, one round trip
agentglass-browser links                       # what this page reaches, deduplicated
agentglass-browser count "[data-testid=row]"   # how many match (omit the selector: interactive count)
agentglass-browser search "shipping"           # find text, get the matches with their hrefs
agentglass-browser interactive                 # what can be acted on: id, role, name, href/value/options
agentglass-browser forms                       # the forms as forms: fields with labels, the submit, loose fields
agentglass-browser attr e17 href data-testid   # one element's attributes (no names: all of them)

All eight are reads, all eight are clamped by --max-tokens and the same redaction seam as everything else, and only extract, search and attr take arguments — the others answer with the whole page in the right shape. extract's answer names the fields that matched nothing, so you never invent a value for a field that was not there. interactive and forms hand out the same ids observe does, so what they list is what the next click or fill takes; a password's value never travels in any of them.

Tools a page offers (WebMCP)

Some pages announce tools of their own (document.modelContext). tools lists them, call-tool NAME --args '{...}' runs one. Both need AGENTGLASS_BROWSER_WEBMCP=1 on the server and answer with a refusal naming the flag without it. The DOM path stays the default: observe, then click or fill. Reach for a page tool only when the page offers one for exactly what you are doing, and go back to the DOM path the moment it fails.