version: "1.0.0" name: ui-tests-migration description: "Create, migrate, and stabilize PowerToys UI tests with Microsoft.PowerToys.UITest.Next and winappcli, through local VM success, commit/push, and CI validation. Use for ports, new UITest projects, flaky CI tests, persistent Hyper-V validation, Settings IPC authentication/test signing, Explorer/Shell selection, preview handlers, thumbnail providers, hotkey activation, stateful process lifecycle, composed WinUI/WebView visual baselines, or foreground failures. Covers APIs, scaffolding, test design, diagnostics, and the required handoff to ui-tests-pipeline-ci. Keywords: UI test, UITests, UITestAutomation.Next, winappcli, WinAppDriver, Selenium, Settings IPC, not-microsoft-signed, Authenticode, local VM, Hyper-V, checkpoint, migrate, flaky, CI stability, Explorer, Shell extension, WebView2." license: Complete terms in LICENSE.txt

PowerToys UI-Tests Migration (legacy → .Next)

Convert a PowerToys module's UI tests from the legacy WinAppDriver / Selenium / Appium harness (Microsoft.PowerToys.UITest, in src/common/UITestAutomation/) to the new winappcli harness (Microsoft.PowerToys.UITest.Next, in src/common/UITestAutomation.Next/).

The new harness shells out to winapp.exe and parses its JSON — no WinAppDriver server on :4723, no Selenium/Appium NuGet packages, no `WindowsElement`/`WindowsDriver`. The public shape (UITestBase, Session, Find<T>, By, element wrappers like ToggleSwitch) is deliberately similar, so most of the work is mechanical API mapping plus reworking a few patterns that don't translate one-to-one (XPath selectors, stateful elements, instance mouse/keyboard helpers).

When to use this skill

Use this skill when the task is to:

• Port a module's existing legacy UI tests to .Next (e.g. "migrate the ScreenRuler UI tests to

the new framework", "convert FancyZones.UITests to winappcli").

• Create a new [Module].UITests.Next project that re-implements the legacy tests with the new

harness, leaving the old project in place.

• Stand up brand-new .Next UI tests for a module that has no UI tests at all, by reading the

module's human test sign-off markdown (e.g. ColorPickerUITest.md) and turning each manual checklist item into an automated test.

• Validate a new or migrated suite in a local Windows VM through an unattended

build/package/deploy/run/TRX/diagnose loop. Use a retained VM for fast iteration and a restored baseline checkpoint when clean-profile behavior matters.

This skill is the how: the framework differences, the API mapping, the project scaffolding, the naming rules, the recurring PowerToys test recipes, and the build/validate loop. The what (which module, which tests) comes from the calling prompt.

End-to-end completion contract

A request to create, migrate, or stabilize UI tests includes the complete delivery loop: implement -> build -> full local VM matrix -> commit and push -> scoped CI -> terminal results. Local success is a handoff, not completion. After the full default and constrained suites pass, invoke ui-tests-pipeline-ci automatically; do not wait for a separate "push" or "run CI" request. Keep commit/push and CI validation as open TODOs from the start.

Respect an explicit local-only/no-push/no-CI request. A read-only investigation, VM setup, or request to run an existing suite locally does not authorize publishing changes. CI remains Microsoft FTE-only and requires the pipeline skill's successful access preflight; report an exact blocker when unavailable, never call local-only evidence CI-validated. The pipeline skill owns publication, queueing, synchronous waiting, and the three-run stabilization limit.

Reference implementation — read these working examples before porting anything. They are

the ground truth for "what good looks like" with each harness:

- New (`.Next`): ColorPickerEndToEndTests.cs

— full end-to-end scenario (navigate Settings → toggle module → read shortcut → fire hotkey →

read overlay → click-capture → inspect editor), driven entirely through winappcli.

- Legacy: TestSpacing.cs

+ TestHelper.cs

— a UITestBase subclass plus a static helper that navigates, toggles, reads the shortcut, fires

the hotkey, and validates the clipboard.

- Worked Scenario-A port (validated 5/5, where the legacy suite scored 0/5 locally): the

ScreenRuler suite ported from the legacy project above lives in

ScreenRuler.UITests.Next/TestHelper.cs

+ 5 test classes. It is the canonical port reference — cross-window toolbar discovery via

Session.FromProcess, a DPI-aware app.manifest, cursor centering, and patient hotkey

activation are all there because real runs needed them (see

references/patterns-and-pitfalls.md).

- Stateful/visual reference (validated 15/15 across Win10 x64, Win11 x64, and ARM64):

PeekFilePreviewTests.cs

demonstrates stable Explorer Shell selection, toggle-hotkey activation, process-preserving

pinning tests, renderer readiness, and composed WinUI/WebView visual baselines.

- Explorer/Shell-extension reference (validated across x64 and ARM64 CI):

FileExplorerAddonsTests.cs

demonstrates class-scoped runner reuse, one-time Shell restart, state-aware Preview pane

activation, exact Shell selection, deterministic icon sizes, provider-log readiness, and

failure media captured before Explorer teardown. Read

references/explorer-shell-tests.md before testing Explorer.

Required reads (in order)

1. This `SKILL.md` — the decision tree (which scenario), the naming rules, the high-level

workflow, and the build/validate loop.

1. [references/framework-differences.md](references/framework-differences.md) — the conceptual

deltas you MUST internalize before writing code: winappcli engine, stateless elements, selector grammar (no XPath/CssSelector), session scopes (window vs process), lifecycle/hygiene/module pre-enablement, multi-window discovery, and what the new harness does NOT (yet) provide.

1. [references/api-mapping.md](references/api-mapping.md) — the line-by-line cheat sheet:

namespaces, By, Element actions/properties, Session, UITestBase, the static Keyboard/Mouse/Clipboard helpers, and the element-wrapper catalog. Keep this open while editing.

1. [references/project-setup.md](references/project-setup.md) — csproj scaffold, naming/placement

rules, .slnx registration, and how to build & run a .Next project. Uses the templates/ starter files.

1. [references/porting-workflow.md](references/porting-workflow.md) — the two end-to-end

playbooks: A) port existing legacy tests, and B) author tests from a human sign-off markdown when none exist.

1. [references/patterns-and-pitfalls.md](references/patterns-and-pitfalls.md) — adaptable recipes

for the recurring PowerToys patterns (toggle a module + verify its process, read the activation shortcut from a ShortcutControl, fire a global hotkey reliably, inspect the clipboard, discover overlay/editor windows) and the gotchas that bite during migration.

1. [references/explorer-shell-tests.md](references/explorer-shell-tests.md) — required for tests

involving Explorer, preview handlers, thumbnail providers, Shell selection, view modes, or Shell restarts. Covers lifecycle boundaries, authoritative signals, and failure evidence.

1. [references/ci-stability.md](references/ci-stability.md) — the CI-stability capstone: the

Win32-window vs UIA-element mental model, state-boundary worksheet, stable-sample waits, retry semantics, foreground/integrity constraints, process lifecycle, composed visual capture, and a pre-flight checklist to apply BEFORE the first CI push. It also covers Release Runner/Settings IPC authentication, the existing CI companion-signing mechanism, and why a visible Settings toggle must never be rescued with a settings-file/restart fallback. Read this to spend one CI iteration instead of six.

1. [ui-tests-local-vm](../ui-tests-local-vm/SKILL.md) — the live desktop execution loop:

scaffold or reuse a persistent Hyper-V VM, run as a true standard user, refresh only changed payloads, iterate through durable TRX/evidence, and restore or recreate the baseline for clean-profile validation.

1. [ui-tests-pipeline-ci](../ui-tests-pipeline-ci/SKILL.md) — the mandatory post-local handoff

for implementation tasks: preflight, scoped commit/push, exact-revision CI, and terminal sign-off.

Pick your scenario

flowchart TD
    A[Module to migrate] --> B{Does a legacy<br/>UITests project exist?}
    B -- Yes --> C["Scenario A: PORT<br/>Create [Module].UITests.Next<br/>Re-implement each legacy test"]
    B -- No --> D{Is there a human test<br/>sign-off .md?}
    D -- Yes --> E["Scenario B: GREENFIELD<br/>Create [Module].UITests<br/>Turn each checklist item into a test"]
    D -- No --> F[Ask the user for the<br/>test spec / sign-off doc]

Place the new project under `src/modules/[Module]/Tests/[Module].UITests.Next/` (or …/Tests/[Module].UITests/ for Scenario B). If the module already keeps tests in a different Tests/ layout, match the module's existing convention rather than forcing this one — see references/project-setup.md.

Keep it abstract. Every PowerToys module is unique and the legacy tests were written by

different people in different styles. Treat the recipes in this skill as adaptable patterns, not

a rigid script. Re-create the intent and assertions of each test; do not mechanically translate

brittle, harness-specific scaffolding (Selenium Actions, XPath walks, manual driver attaches) when

the new harness has a cleaner idiom.

High-level workflow

Create a TODO list and work top-to-bottom. Each step links to the reference that drives it.

-[ ] 1. Identify the module + scenario (A port / B greenfield) — this SKILL.md "Pick your scenario"
-[ ] 1a. Read the module's developer docs — `doc/devdocs/modules/<module>.md` (if the exact file is
        missing, search `doc/devdocs/`, including `doc/devdocs/common/`) — to learn its
        development-cycle specifics BEFORE writing tests: how its shell extensions / context menus
        register, whether they need a **Release** build (`NDEBUG`) or a **signed** sparse MSIX package,
        and any Explorer-restart or first-run needs. Skipping this produces opaque failures — e.g. a
        context-menu entry never appears because a Debug build compiles registration out, or an
        unsigned `.msix` fails to register (`0x800B0100`).
-[ ] 2. Read the two reference examples (ColorPicker .Next + ScreenRuler legacy) end-to-end
-[ ] 3. Inventory the source:
        • Scenario A → list every [TestMethod] + shared helper in the legacy project
        • Scenario B → read the module's sign-off .md; list each manual checklist item
  • For each workflow → list every external boundary (runner, Explorer, HWND, renderer,
    compositor, child process) and its authoritative ready signal
        — references/porting-workflow.md
-[ ] 4. Internalize the deltas — references/framework-differences.md
-[ ] 5. Scaffold the new project (csproj + PerMonitorV2 app.manifest from templates, name per the
  table, register in .slnx)
        — references/project-setup.md
-[ ] 6. Re-implement tests, mapping each API as you go — references/api-mapping.md
        +recipes from references/patterns-and-pitfalls.md
-[ ] 6a. If Explorer/Shell is involved, apply references/explorer-shell-tests.md
-[ ] 7. Apply the CI-stability checklist BEFORE building — references/ci-stability.md
  (stable authoritative signals, retry classification, foreground/integrity, lifecycle reset
  scope, non-activating helper processes, composed capture, DPI manifest, single-module enable,
  first-run suppression)
-[ ] 7a. If a test changes a module's enabled state through Settings, keep the real Settings UI +
  immediate runtime assertion. Verify the selected UITest project is covered by the existing
  `$requiresAuthenticatedSettingsIpc` companion-signing path in
  `.pipelines/v2/templates/job-test-project.yml`; never add a test-side settings/restart fallback
  for Release CI — [references/ci-stability.md](references/ci-stability.md#principle-5a--keep-module-lifecycle-tests-on-real-release-settings-ipc)
-[ ] 8. Build the new project to exit code 0 — this SKILL.md "Build & validate"
-[ ] 9. Run one deterministic test in the local VM and diagnose the first failure
      — ../ui-tests-local-vm/SKILL.md
-[ ] 10. Rerun the focused test after each fix, then widen to the complete module suite with bounded
  timeouts; parse TRX and verify durable evidence export
-[ ] 11. If the local VM is unavailable or unsupported, run on another live desktop or report the exact
   environmental blocker; do not silently stop at compile validation
-[ ] 12. Complete the full default and Constrained suites on both guest OSes, plus applicable
   architecture builds/guests; preserve counts, payload hashes, and evidence
-[ ] 13. Invoke ui-tests-pipeline-ci, pass its access preflight, commit only task-owned changes,
   and push the feature branch; record the exact SHA
-[ ] 14. Preview and queue scoped CI, persist its build ID, wait synchronously, and diagnose/retry
   within the three-run ceiling; finish only on verified terminal success or an explicit blocker

Build & validate

The .Next harness needs winapp.exe only at run time, not build time — the project has zero managed dependency on the engine. So you can always compile-verify a migration even on an agent with no winappcli installed.

# 0. FIRST build of a brand-new project: restore so the assets file exists, otherwise the build
#    fails with NETSDK1004 "Assets file ... project.assets.json not found".
dotnet restore src\modules\<Module>\Tests\<Module>.UITests.Next\<Module>.UITests.Next.csproj -p:Platform=x64
#    (Equivalently, run tools\build\build-essentials.cmd once at the start of the session.)
 
# 1. Build just the new test project (fast inner loop). Prefer the repo build script.
tools\build\build.cmd -Path src\modules\<Module>\Tests\<Module>.UITests.Next -Platform x64 -Configuration Debug
#    Exit code 0 = success; non-zero = failure. On failure read the errors log next to the project:
#    build.<Configuration>.<Platform>.errors.log
#    Do not substitute `dotnet build` when UITestAutomation.Next's COM references are in the graph:
#    .NET SDK MSBuild cannot run ResolveComReference and fails with MSB4803. Use the repo script or
#    Visual Studio's full-framework MSBuild.exe; use `dotnet restore` only to create project.assets.json.
 
# 2. Run (needs a live desktop). A .Next project is a Microsoft.Testing.Platform Exe — run the
#    produced exe directly with a TRX report; filter to one test/category for a tight loop.
$exe = "<repo>\x64\Debug\tests\<Module>.UITests.Next\net10.0-windows10.0.26100.0\<Module>.UITests.Next.exe"
& $exe --filter "TestCategory=<Cat>" --report-trx --report-trx-filename run.trx --results-directory <dir>
#    --filter accepts "TestCategory=X" or "FullyQualifiedName~Y"; omit it to run everything.
#    Exit 0 = all passed. Parse the .trx for per-test outcomes + failure messages.

• **Default to persistent local VM validation —

ui-tests-local-vm.** It keeps the interactive desktop and staged tools, refreshes only changed archives, and returns durable status/TRX/evidence. Do not modify stabilized tests merely to improve a VM-specific pass rate when the task only asks whether the execution loop works. Finish clean-profile claims from a restored known baseline or a fresh named VM volume.

• Design for CI stability up-front — [references/ci-stability.md](references/ci-stability.md).

Before the first push, walk its pre-flight checklist (authoritative-signal retries instead of fixed sleeps, navigation via UIA invoke, interaction-scoped foreground checks, non-activating helpers, Win32 window/overlay detection, screen-capture cold-start handling, DPI manifest, single-module enable, first-run suppression). Most "passes local, fails CI" loops come from skipping one of these; applying them proactively is how you spend one CI iteration instead of six.

• Run it in a loop: write → build → run → diagnose → repeat. UI tests surface environment-real

failures (DPI scaling, cursor position, hotkey-arming races) that only a live run reveals. Start with one deterministic test (e.g. the activation/toggle test), get it green, then widen.

• Diagnose from the artifacts, not from the assertion message. Every failed test attaches a

desktop screenshot, and in pipeline mode an MP4 of the run. Open them before forming any theory — especially before concluding the product is broken. An assertion can only say "found 0 rows"; the screenshot says whether the list was empty or whether your selector was wrong. This is the single highest-leverage habit in the agentic loop: skipping it cost ~8 iterations and a confident but entirely wrong product-defect report on File Locksmith (see references/patterns-and-pitfalls.md Pitfall 26). If there is no video, find out why rather than proceeding blind — the harness now prints the reason (a clean Windows image without the Visual C++ redistributable cannot load the native encoder).

• First, run the *legacy* suite once for a baseline — and run it ELEVATED. The legacy harness

launches PowerToys via ProcessStartInfo { Verb = "runas" } (elevated), so a non-elevated test host can't complete the launch and every test fails at startup with a misleading `Win32Exception` cascade — a false 0/N that looks like "the tests are broken" but is purely the run method. (That's why VS Test Explorer passes them: VS runs as admin.) Run from an elevated terminal: start WinAppDriver.exe on 127.0.0.1:4723, then run the built DLL with vstest.console.exe (see references/porting-workflow.md §A0 for the -Verb RunAs recipe). A measurement failure on a scaled (non-100%) display is usually a pre-existing DPI issue (Pitfall 12), not something the port must reproduce — the ScreenRuler legacy suite scores 4/5 elevated here (Bounds fails at 150% scale) while the .Next port scores 5/5. .Next tests themselves need no elevation (the new harness launches the runner non-elevated).

• Always build to exit code 0 before proceeding to live validation. Fix every compile error — do

not leave // TODO: port this stubs that break the build.

• Running the tests requires a live interactive desktop plus winapp.exe

(winget install Microsoft.winappcli, or set WINAPP_CLI_PATH). The whole PowerToys runner is launched by the harness (PowerToys.exe --open-settings) — you should see the Settings window appear. If no supported live desktop is available, report local validation blocked, along with the build result and checklist coverage. Compile-only evidence does not complete an implementation task or satisfy the CI local gate.

• New .csproj files under src/ MUST <Import Project="$(RepoRoot)src\Common.Dotnet.CsWinRT.props" />

right after <Project Sdk=...> (CI audits this). The template already does.

What NOT to do

• Do NOT delete or edit the legacy `[Module].UITests` project in Scenario A. The .Next project

lives alongside it; removing the old one is a separate, explicit decision for the maintainers.

• Do NOT change product behaviour. This is a test-only migration. The one sanctioned product edit

is adding `AutomationProperties.AutomationId` to a control that is otherwise unaddressable — an icon-only button whose label lives in a tooltip has no UIA Name at all, and the alternative is a brittle coordinate click. Use AutomationProperties.AutomationId, never x:Name (which also emits a code-behind field); see references/patterns-and-pitfalls.md Recipe 16. Anything larger — a hidden automation-peer TextBlock, a new property, a state string — must be flagged for the user instead. (ColorPicker's ColorHexAutomationPeer hook is a documented, pre-existing exception — see its class remarks.)

• Do NOT port the legacy plumbing literally. No Selenium Actions, no WindowsDriver/WindowsElement,

no By.XPath/By.CssSelector, no :4723. Map them to the winappcli idioms in references/api-mapping.md.

• Do NOT add a `ProjectReference` to `UITestAutomation.csproj` (the legacy harness) — reference

`UITestAutomation.Next.csproj` only.

• Do NOT invent assertions for a vague sign-off item. If a checklist line has no observable

pass/fail signal, implement what you can and leave a clearly-marked TestContext.WriteLine note (or skip with an explanation) rather than asserting on something you can't actually read.

• Do NOT introduce new third-party NuGet dependencies. The .Next harness is intentionally

dependency-free (MSTest only). Use the Win32-based helpers it already ships.

• Do NOT retry a toggle hotkey blindly. Once any target window appears, wait for initialization;

resending the chord may close a healthy window. Restart only after a terminal readiness failure.

• Do NOT replace or weaken visual baselines before proving capture is correct. Foreground HWND,

DWM z-order, composed WebView content, theme, and platform are separate failure sources.

• Do NOT work around a Release Runner `not-microsoft-signed` Settings rejection in test code. Do

not seed the global enabled map, restart PowerToys, bypass authentication, or weaken the lifecycle assertion. Reuse the pipeline's existing Runner/Settings companion-signing mechanism; see references/ci-stability.md.

What is NICE to do

• Improve the framework when a helper is demonstrably reusable. Prefer small composable APIs

(WaitHelper, WindowControl, ExplorerShell) over module-specific mega-helpers. Keep product semantics such as Peek pin-state preservation in the module test.