lahfir/agent-desktop/agent-desktop-windows

agent-desktop-windows

The observe-act loop, ref system, JSON envelope, sessions and tracing are the same on every platform. This file covers only what differs on Windows.

Requires Windows 10 1809+ / Windows Server 2019+ (x64 or ARM64). Run it in the interactive desktop session of the signed-in user. Session 0 (services and SYSTEM tasks), Server Core, the secure desktop (UAC prompts, the sign-in screen), a locked desktop and another user's session are not supported: observation, actions and capture there fail or return nothing.

Capability Table

First Contact

• Quote every ref in PowerShell. PowerShell reads a bare @token as its

splatting operator and deletes the argument before the binary sees it, so set-value @s8f3k2p9:e1 hi fails INVALID_ARGS. Write set-value '@s8f3k2p9:e1' hi. cmd.exe and bash need no quoting.

• A window behind another window is drivable. A minimized one is not.

Every element of a minimized window reports offscreen and ref actions fail. Run restore --app <image> and snapshot again. Do not use focus-window by reflex: it steals the user's foreground.

• No permission dialog exists. permissions reports automation as

not_required. Elevation boundaries are the only access control.

• Chromium and Electron apps read thin before they settle. Pass a longer

--timeout-ms to snapshot instead of concluding the tree is empty.

• Blocked shortcuts. alt+f4, win+l, win+d, alt+tab, win+r,

win+x, win+e, ctrl+shift+esc and their modifier supersets are refused without --force. In combos, cmd and meta are the Windows key, so meta+c is a shell shortcut. Write ctrl+c to copy.

• `type` needs `--headed`. It is keystroke synthesis through SendInput,

so a strict-headless type returns POLICY_DENIED. Use set-value, or pass --headed. It ends delivered_unverified (verification_scope: "unavailable") when the field exposes no readable Text selection, and can verify when one is readable: confirm with a fresh get.

• `press --app` writes to the foreground queue with no per-process

targeting. A headless press to a non-frontmost app fails closed. A non-interactive caller (service, scheduled task, CI job) gets delivered_unverified.

• `launch` takes an absolute path or a bare name under System32 or the

Windows directory, with or without .exe (notepad or notepad.exe), not a display name. Use list-apps to find a running app.

• Notifications: list-notifications with the Action Center closed needs

--headed. Single-item dismiss-notification and notification-action need --headed and --expected-app or --expected-title for the entry at INDEX.

• Surfaces: an open or context menu is the menu surface. Windows does

not advertise a menubar surface.

• Commands that raise shell chrome take the foreground. Strict headless

refuses them with POLICY_DENIED before anything is raised.

• Stateful actions run once, then verify by re-reading. delivered_verified

means the readback matched. ACTION_FAILED with details.kind: "post_action_verification" means the action landed but the readback differs: read details.post_state and do not retry. A set-value the app reformats reads as failed. See references/troubleshooting.md.

• Unsigned binary warnings are expected. See references/troubleshooting.md.

What differs on Windows

• Ref actions are semantic UI Automation calls in headless and --headed mode, except right-click, double-click, triple-click, type and focus, which are physical or take focus and need --headed.
• The cursor overlay plays its per-action travel only for headless semantic actions. A --headed action moves the real cursor instead.

References

• references/shell-and-overlay.md: open it for open-system-surface, tray

and taskbar surfaces, notifications, the cursor overlay, hosted (UWP) window identity and menu detection coverage.

• references/permissions-and-elevation.md: integrity levels, UIPI, blocked

combos, protected processes, the cross-process interaction lease.

• references/chromium-and-electron.md: settle timing, covered-window hazard,

WPF wrong-provider trap, unnamed web content.

• references/troubleshooting.md: symptom-to-cause map, action verification

outcomes, saving a document headless, install warnings.

Read any file from the binary: agent-desktop skills get agent-desktop-windows references/troubleshooting.md.