zpoint/vibe-seller/browser-harness

browser-use 0.13 changed everything. There are no subcommands. You

no longer run browser-use open <url>. Instead you pipe Python helper code

to browser-use via a heredoc; helpers are pre-imported and a background

daemon is attached automatically.

Store Tasks: browser-use is a per-store wrapper script that

auto-injects BU_NAME (the store session) and BU_CDP_WS (the store's CDP

proxy). You cannot set BU_NAME, BU_CDP_URL, BU_CDP_WS, or --mcp

yourself — the wrapper blocks them. Use the default session for the store's

seller center, or --session <slug>-aux for non-seller-center sites (the

wrapper maps this to the aux session; no other --session value is

allowed).

No-store (orchestrator) Tasks: browser-use is the store-less web

wrapper (bin/_web). Use it only for neutral public web work (search,

tracking/logistics, research) — NEVER for a store's seller center or to log

into store/platform accounts (create a store sub-task for those).

Browser Automation with browser-use (0.13, heredoc interface)

Drive the browser by piping Python to browser-use. Helpers are pre-imported; the harness calls ensure_daemon() before running your code, so the browser attaches automatically to the store's CDP endpoint.

browser-use <<'PY'
new_tab("https://example.com")   # first navigation is new_tab(), NOT goto
wait_for_load()
print(page_info())
PY

The wrapper takes the heredoc form only. Put every statement inside the heredoc.

Prerequisites

browser-use --doctor    # verify installation / CDP connectivity

Budget every invocation: the wrapper kills it at 120 s

The store wrapper runs the real browser-use under a 120-second alarm, and a timeout is treated as evidence the browser is wedged: the wrapper bumps a strike counter, force---reloads the daemon, and on the second consecutive timeout prints UNRECOVERABLE, exits 75, and tells you to stop retrying and report a gap. Any other outcome — even an ordinary Python error — resets the counter.

A slow-but-healthy call is indistinguishable from a wedged browser. So a legitimately long heredoc does not merely get cut off: two of them in a row will convince the wrapper (and then you) that a perfectly good browser is dead, and the honest-reporting rule then makes you abandon real work. Keep each invocation comfortably under the limit:

• One unit of work per invocation — one page, one SKU, one export.

Drive the loop from the shell, not inside the heredoc, and append results to a file under /tmp/<task>/ so progress survives.

• Count your polls. A render-wait of range(15) with sleep(3) is

45 s on its own; two of those plus navigation overruns 120 s.

• If you genuinely need a long single operation (a slow export), poll it

across separate invocations rather than sleeping inside one.

A timeout also leaves the tab mid-operation, so re-read state at the start of the next invocation instead of assuming where you left off.

Core Workflow

1. Navigate: new_tab(url) — for the first page **and every later

navigation. There is no page object** in the heredoc scope, so page.goto(url) raises NameError; use new_tab(url) (or click a link) to move around.

1. Understand state — via the DOM (PREFERRED, no vision needed):

page_info() for page-level facts (url/title/size), and `js(...)` to read the DOM — text content AND every element's on-screen coordinates via getBoundingClientRect (see "Locate & click without vision"). This is the primary way to drive the browser; a non-vision model completes the whole flow this way.

1. See the layout (OPTIONAL — vision models only):

capture_screenshot() returns a PNG path (~/.vibe-seller/bh-tmp/<session>/shot.png, overwritten each call); print() it and Read that PNG to view it. It is a Python helper — call it inside the browser-use <<'PY' heredoc; typed as a shell command it is a bash parse error, and an ls of the dir then shows an older picture, not this page. Use it only to disambiguate a crowded layout — never depend on it. If your model can't view images, skip screenshots entirely and use step 2.

A screenshot is not a data source. Never take a value you will act on or report — an ID, a number, a row, a column that exists, a count — from an image. Read it from the DOM with js(...). A screenshot answers "where is it on screen", nothing else.

This is not caution about edge cases; it is the observed default. On a page holding exactly two rows, models on two different providers each described a table that did not exist — invented columns (ACOS, CPC, Bid), invented campaigns, invented statuses — then spent a dozen turns hunting for the data they had "seen", and one named a nonexistent campaign in its final report to the user. Nothing was wrong with the PNG. When the DOM and the picture disagree, the DOM is right and the picture is a hallucination — do not go looking for the difference, and never reconcile by trusting the image.

1. Interact: get an element's centre coords from step 2, then

click_at_xy(x, y); set input values with js(...). Re-read with page_info() / js(...) after to confirm.

1. After navigation: wait_for_load(); if the tab is stale/internal,

ensure_real_tab().

Helper API

Helpers are pre-imported into the heredoc namespace:

new_tab(url)  # open a new tab and navigate (use for EVERY navigation)
page_info()  # structured summary of the current page
capture_screenshot()  # → PNG path (~/.vibe-seller/bh-tmp/<session>/shot.png); LAYOUT only, never data
click_at_xy(x, y)  # click at pixel coordinates
wait_for_load()  # wait for navigation/network to settle
ensure_real_tab()  # switch off a stale/internal (chrome://) tab
js('<javascript>')  # run JS; returns the SERIALIZABLE result only
cdp('Domain.method', **params)  # raw CDP — params are KEYWORDS, not a dict
# e.g. cdp('Page.navigate', url='...')
drain_events()  # → and clear the buffered CDP events (see file upload, below)

• `js()` returns serializable values only. js("document.title") and

js("return 1+1") work; but js("document.querySelector(...)") returns None (a DOM node can't serialize). Return numbers, strings, or plain objects/arrays — e.g. an element's coordinates (below), not the element itself. For an element reference (to set a file input) use cdp('Runtime.evaluate', expression=..., returnByValue=False) → objectId (note: cdp() params are keyword args, never a positional dict — see "Uploading a file").

• `js()` does NOT parse JSON — so do not `JSON.stringify` your result.

The value comes straight from CDP returnByValue, so the JS type maps to the Python type: return [{a:1}] gives you a list of dicts already, while return JSON.stringify([{a:1}]) gives you a `str` that you must then json.loads yourself. Both directions bite: stringifying and not parsing makes for row in data iterate characters; not stringifying and then parsing raises TypeError: the JSON object must be str, bytes or bytearray, not list. Return the object directly and use it as-is. If you inherit code whose shape you can't be sure of, normalise once rather than guessing:

```python import json # stdlib modules: import them, don't rely on the namespace

def jsjson(expr): d = js(expr) return json.loads(d) if isinstance(d, str) else d ```

Only the helpers above are guaranteed pre-imported. json, time and friends do happen to be reachable in the heredoc namespace today (they leak in through the harness's own import *), but that is an implementation detail of the wheel, not a contract — one __all__ upstream and it stops. Import the stdlib you use.

Locate & click an element WITHOUT vision (the preferred path)

You do not need to see the page. Read the DOM and compute click coordinates from getBoundingClientRect, then click_at_xy. This drives any page — buttons, links, shadow-DOM kat-* components — with no screenshot:

browser-use <<'PY'
# one element by selector → its centre coords + text (None if not found):
box = js("""
  var el = document.querySelector('button.submit');   // any CSS selector
  if(!el) return null;
  var r = el.getBoundingClientRect();
  return {text:(el.innerText||el.value||'').slice(0,40),
          x:Math.round(r.x+r.width/2), y:Math.round(r.y+r.height/2)};
""")
print("target:", box)
if box: click_at_xy(box["x"], box["y"])
 
# OR enumerate all clickables to find the right one by its text:
els = js("""
  return [].slice.call(document.querySelectorAll('a,button,input,[role=button],kat-button'))
    .map(function(el){var r=el.getBoundingClientRect();
      return {text:(el.innerText||el.value||'').slice(0,40),
              x:Math.round(r.x+r.width/2), y:Math.round(r.y+r.height/2)};})
    .filter(function(e){return e.x>0 && e.y>0;});
""")
print(els)          # pick the one whose text matches, then click_at_xy(it.x, it.y)
PY

A plain `<a href>` is a navigation, not a click. Read the href and new_tab(href). Clicking one means landing inside the anchor's own rect, and an anchor is usually a small target inside a much larger parent: the centre of the table row or cell holding it is typically not on the anchor at all, so click_at_xy hits dead space and silently does nothing — no error, no navigation, and page_info() still shows the old page. Reserve clicking for controls that have no href (buttons, kat-*, JS handlers).

A control BELOW the fold that won't scroll into view

Some pages (e.g. Amazon's "Generate Spreadsheet" popover) put the button you need below the viewport, inside a kat-popover/panel that scrollTo/scrollIntoView can't bring up — the window is short (e.g. 839px tall) and click_at_xy can't hit an off-screen y. Don't fight the scroll: grow the layout viewport with CDP so the whole panel fits, then click the (now on-screen) coordinate, then restore:

browser-use <<'PY'
import time
cdp("Emulation.setDeviceMetricsOverride",
    width=1920, height=2400, deviceScaleFactor=1, mobile=False)  # tall viewport
time.sleep(1)
box = js("""
  var els=document.querySelectorAll('kat-button,button,[role=button]');
  for(var i=0;i<els.length;i++){var t=(els[i].innerText||els[i].getAttribute('label')||'').trim();
    if(/generate spreadsheet/i.test(t)){var r=els[i].getBoundingClientRect();
      return {x:Math.round(r.x+r.width/2), y:Math.round(r.y+r.height/2)};}}
  return null;
""")
if box: click_at_xy(box["x"], box["y"])   # trusted click, now on-screen
time.sleep(3)
cdp("Emulation.clearDeviceMetricsOverride")   # restore the real viewport
PY

(Verified: the override raises window.innerHeight to the set value, so a below-fold button becomes reachable; clearDeviceMetricsOverride undoes it. A plain JS .click() still no-ops on kat-* — you need the trusted click_at_xy.)

For an element inside an open shadow root (Amazon kat-*), pierce it in the selector: document.querySelector('kat-file-upload').shadowRoot.querySelector('input'). Set an input's value with js("document.querySelector('#q').value='kettle'") (then click its search icon — some inputs need the click to fire events).

Only the helpers above (plus Python builtins) are in scope — the heredoc runs as a plain Python script. `import` every stdlib module you use (`time`, `json`, `re`, …). Bare sleep 3 is a SyntaxError. Prefer `wait_for_load()` over sleeping — reach for import time; time.sleep(n) only when you must wait on something wait_for_load() can't observe (e.g. an async in-page render after a click).

Multiple statements run in one heredoc (this replaces && chaining):