d-wwei/browser-control-skill/browser-control

if command -v npx &>/dev/null; then
  _UK_DIR="${SKILL_DIR:-$(cd "$(dirname "$0")" && pwd)}"
  _UK_JSON=$(npx update-kit check --json --config "$_UK_DIR/../../update-kit.config.json" 2>/dev/null || echo '{}')
  _UK_KIND=$(echo "$_UK_JSON" | python3 -c "import sys,json; print(json.load(sys.stdin).get('kind',''))" 2>/dev/null || echo "")
  if [ "$_UK_KIND" = "upgrade_available" ]; then
    _UK_VER=$(echo "$_UK_JSON" | python3 -c "import sys,json; print(json.load(sys.stdin).get('latest',''))" 2>/dev/null)
    echo "Browser Control Skill update available: $_UK_VER — run: cd ~/.claude/skills/browser-control && npx update-kit apply"
  fi
fi

Browser Control

Control the user's real Chrome browser with full login session inheritance. Multi-channel web access — from lightweight search to full CDP browser automation — with three-layer safety, sub-agent parallel dispatch, and site experience memory.

When to Use

• Access any web page — public or behind corporate authentication (SSO, MFA, security gateways)
• Search, fetch, and extract information from the web
• Automate browser interactions: clicking, form filling, navigation, file upload
• Read content from dynamic pages (SPA, lazy-load, infinite scroll, virtual DOM)
• Write operations: compose messages, submit forms, upload files on web apps
• Rich interactions: React/Vue apps, rich text editors, file pickers, dropdowns
• Parallel research: dispatch sub-agents to investigate multiple targets simultaneously

Companion Sub-Skill: Web Search

For enhanced public web search, the companion web-search sub-skill (powered by omni-search-skill) provides multi-engine search, fetch, resolve, and crawl — stronger than built-in WebSearch/WebFetch.

On first use, check availability:

OMNI_SEARCH_DIR="${OMNI_SEARCH_DIR:-$(find ~ -maxdepth 4 -type d -name 'omni-search-skill' 2>/dev/null | head -1)}"
[ -n "$OMNI_SEARCH_DIR" ] && [ -f "$OMNI_SEARCH_DIR/scripts/omni_search.py" ] && echo "available" || echo "not installed"

If not installed, suggest once:

💡 For enhanced web search, consider installing omni-search-skill:

git clone https://github.com/d-wwei/omni-search-skill.git && cd omni-search-skill && python3 -m pip install -r requirements.txt

Do not block on this — built-in multi-channel tools (WebSearch → WebFetch → Jina → curl → CDP) work without it. Only suggest once per session.

Browsing Philosophy

Browse like a human — goal-driven, adaptive, evidence-based.

Do not over-plan steps upfront. Enter with a clear goal, observe the page, decide the next action based on what you see, handle obstacles as they arise, and dig deeper when content is insufficient. Every decision traces back to: "Am I getting closer to what the user needs?"

The Four Phases

1. Understand — Clarify what the user wants. Define success: what information, what action, what result? This anchors all subsequent decisions.

2. Enter — Choose the most direct entry point for the task. Consider platform characteristics and success criteria. If login is needed or static methods are known to fail (e.g., Xiaohongshu, WeChat), go straight to CDP.

3. Verify — Every result is evidence, not just pass/fail. Compare against the success criteria from phase 1. Is the path progressing? Is the quality, relevance, and completeness pointing toward the goal? If direction is wrong, pivot immediately — don't retry the same approach hoping for different results. Search misses don't always mean "wrong query" — they might mean "target doesn't exist." API errors, missing DOM elements, unchanged results after retry: these all signal "reassess direction."

4. Complete — Check against the defined success criteria. Stop when done. Don't over-operate for the sake of "completeness."

Step-by-Step Reasoning (EOPA)

Before each browser action, follow this mental model:

E — EVALUATE: What happened after my last action? Did it succeed?

• If I navigated: did the URL change? Is the page loaded?
• If I clicked: did anything change on the page?
• If I typed: is the text visible in the field?

O — OBSERVE: What does the current page state tell me?

• Read page content or check the current URL
• What interactive elements are available?
• Am I on the right page for the user's goal?

Auto-snapshot rules — decide whether to re-read:

• Last action was read-only (get text, get URL, list elements) → page state unchanged, skip re-read
• Last action was click / navigate / submit / scroll-that-loads → page state likely changed, must re-read
• You already have the exact selector or element index → operate directly, no need to list all elements first
• Never execute the same read command twice consecutively

P — PLAN: What is the single next action needed?

• Do I need to scroll to reveal more content?
• Do I need to wait for something to load?
• Which specific element should I interact with?

A — ACT: Execute exactly ONE browser action, then return to E.

Multi-Channel Web Access

Choose the lightest tool that gets the job done. Escalate only when lighter tools fail.

Jina usage: Prefix URL without http(s):// — e.g., curl -s https://r.jina.ai/example.com. Great for articles, blogs, documentation. May extract wrong sections on dashboards or product pages.

Escalation principle: When multiple search attempts show no quality improvement, escalate to primary sources (official sites, original pages) via CDP.

Information integrity: Search engines are discovery tools, not proof. Primary sources (official sites, original documents, source code) outweigh secondary reports. Multiple media citing the same error creates circular confirmation — always trace to origin.

Programmatic vs GUI Interaction

Inside the browser, two approaches:

• Programmatic (construct URL, eval DOM): Fast and precise when it works, but sites may detect non-human behavior and trigger anti-scraping.
• GUI interaction (click buttons, fill inputs, scroll): Designed for humans — sites never restrict normal UI operations. Reliable fallback.

When programmatic access fails, GUI interaction is the dependable fallback. Site-generated links (DOM hrefs) carry full context; manually constructed URLs may miss implicit parameters.

Platform Detection

Before executing any command, detect the platform:

uname -s

• Darwin → macOS → AppleScript (basic) + CDP Proxy (advanced)
• Linux → CDP Proxy
• MINGW* / CYGWIN* / MSYS* / Windows_NT → CDP Proxy (+ optional agent-browser)

Preflight Check

macOS AppleScript Preflight

# Check Chrome is running and JS access is enabled
osascript -e 'tell application "Google Chrome" to get name'
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "document.title"'

If the second command fails: Chrome → View → Developer → Allow JavaScript from Apple Events (中文: 查看 → 开发者 → 允许 Apple 事件中的 JavaScript).

CDP Preflight (All Platforms)

bash "${SKILL_DIR}/scripts/check-deps.sh"

The script checks Node.js 22+, Chrome debug port, and starts the CDP Proxy if needed.

Chrome remote debugging setup: Open chrome://inspect/#remote-debugging in Chrome, check "Allow remote debugging for this browser instance", restart Chrome if needed.

Mode Routing

This skill operates in two modes. The /browse command handles routing automatically, but the logic is documented here for all agents.

Two Modes

Foreground (`here`) — macOS AppleScript, operates on user's visible page

• Targets front window → active tab — the page the user is looking at
• User sees every action in real time
• Zero dependencies beyond Chrome
• Single-page, supervised operations

Background (`bg`) — CDP Proxy, operates in background tabs

• Creates its own tabs, never touches user's tabs
• User continues browsing normally, zero disruption
• Supports sub-agent parallel dispatch
• Required for advanced write operations (keyboard, upload, hover, isTrusted)

Routing Decision

Platform is macOS?
├─ NO → background (CDP only option)
└─ YES
    ├─ User references current page? ("这个页面", "current tab", etc.)
    │   → foreground
    ├─ Multiple targets or parallel work? ("同时", "simultaneously", etc.)
    │   → background
    ├─ Implies don't-disturb? ("后台", "background", etc.)
    │   → background
    ├─ Needs keyboard/upload/isTrusted/hover?
    │   → background
    └─ DEFAULT → foreground (simpler, zero-dep)

Escalation Within Foreground Mode

If a foreground (AppleScript) operation fails:

1. Try React-compatible event dispatch (still AppleScript)
2. Still fails → auto-start CDP Proxy, switch to CDP for that operation
3. Simple reads can stay on AppleScript even after one operation escalated

Operation-Level Reference

Element Targeting Priority

macOS (AppleScript)

1. Element index — inject List Interactive Elements, then use window.__interactiveElements[index] (most reliable)
2. Text content match — most readable for buttons/links
3. CSS selector — for form inputs, specific elements
4. Coordinates — last resort, fragile across screen sizes

CDP (All Platforms)

1. CSS selector via /click — simple JS click, fast
2. CDP mouse event via /clickAt — trusted, bypasses anti-automation
3. JS eval — for complex targeting logic
4. Coordinates via CDP Input.dispatchMouseEvent — when all else fails

macOS AppleScript Quick Reference

Zero-dependency browser control via macOS native AppleScript. Full reference: modules/applescript-commands.md

# Get current URL
osascript -e 'tell application "Google Chrome" to return URL of active tab of front window'
# Get page text
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "document.body.innerText.substring(0, 15000)"'
# Open URL in new tab
osascript -e 'tell application "Google Chrome" to tell front window to make new tab with properties {URL:"https://example.com"}'
# Click by CSS selector
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "document.querySelector(\"#submit-btn\").click()"'
# Execute arbitrary JavaScript
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "YOUR_JS_CODE_HERE"'

For JXA tab targeting and MutationObserver-based element waiting, see modules/advanced-applescript.md.

CDP Proxy Quick Reference

HTTP-to-CDP bridge proxy for the user's daily Chrome. Full reference: modules/cdp-proxy-api.md