thesoftwarehouse/copilot-collections/tsh-ui-verifying

UI Verification

Verification process, criteria, and tolerances for comparing UI implementations against Figma designs.

Default to asking when anything is off — this is a judgment rule, not a checklist. Every specific blocker named in this skill (missing Figma, auth redirect, wrong page, missing or partial artifacts, unconfirmed URL, tool error, …) is only an EXAMPLE of one underlying rule: whenever you cannot run a real, complete verification against the full artifact base — because something is missing, broken, ambiguous, inconsistent, or simply unexpected, including situations not listed anywhere here — stop and raise it through vscode/askQuestions (when that tool is available to you). Do not guess, do not improvise a workaround, do not fabricate values, and do not proceed on partial evidence. Think about whether the evidence you actually have supports a verdict; if it does not, ask instead of pushing forward.

Verification Process

Use the checklist below and track your progress:

Progress:
- [ ] Step 1: Validate inputs
- [ ] Step 2: Get EXPECTED from Figma
- [ ] Step 3: Get ACTUAL from implementation
- [ ] Step 4: Compare using verification categories
- [ ] Step 5: Generate report

Step 1: Validate inputs

Before starting verification, confirm:

• Figma URL is available for the component/section being verified
• Dev server URL is a user-confirmed pinned session input:
• Standalone verification without a caller-provided URL: on the first verification in a session, ask the user to confirm the exact full dev server URL that should be used for verification. Do not infer it from project config, running processes, port scans, or other discovery.
• Delegated verification with a caller-provided user-confirmed URL: use that exact full URL unchanged for the entire session. Do not rediscover it, normalize it, swap ports, inspect config to suggest another URL, or launch a different local app/server.
• Once the user has confirmed the URL, every downstream verification and capture pass must treat it as pinned session state.
• Dev server is running and the target page is reachable through the CLI capture flow using that confirmed URL:
• Never circumvent an authentication, login, or access/permission gate by any means or technique — proactively or reactively. Legitimate authentication through the app's real login UI is allowed only when the user has explicitly authorized it and supplied the exact inputs required to perform it through a local env-based contract derived from the real login form, direct in-browser entry, or a real storage-state path created from a prior login. Navigate to the pinned URL only as an ordinary user would. Do not assume, fabricate, simulate, seed, inject, or manufacture any signed-in or authorized state. If you notice the gate is trivially bypassable (for example it can be satisfied entirely client-side), report it as a potential security vulnerability when you raise the blocker, so the user is made aware and can plan a fix — flag the concern, never exploit it.
• Use the CLI capture flow to open the page at the full target URL before verification begins
• If the capture flow reports a redirect to a login/authentication screen and the required login inputs were not already provided: first determine whether the redirected screen is a standard credential form. If it is, derive one repo-root .env var per required field from the live form using this order of precedence for the field key: name -> autocomplete -> id -> visible label text. Normalize the chosen key to uppercase snake case and prefix it with TSH_UI_LOGIN_. Examples: email -> TSH_UI_LOGIN_EMAIL, userName -> TSH_UI_LOGIN_USER_NAME, company-code -> TSH_UI_LOGIN_COMPANY_CODE. If the verifying agent has vscode/askQuestions, the immediate next action MUST be a vscode/askQuestions call telling the user to add those exact derived env vars to repo-root .env and confirm when the file is saved. On the next capture pass, reload .env and reuse those env vars without printing their values. Use a prepared storage-state path or direct manual login only when the redirected screen is not a standard credential form (for example SSO chooser, MFA challenge, or captcha), the runtime cannot derive the field keys reliably, or the user explicitly prefers one of those fallbacks. Do NOT bypass, seed, inject, or fake authentication yourself by any means or technique, and never fake an identity or assume a role, even if you can see how the auth check works.

Use this wording pattern for the user message: "The page redirected to login. Add these exact vars to repo-root .env and tell me when the file is saved:

• [DERIVED_ENV_VAR_1]=...
• [DERIVED_ENV_VAR_2]=...

After you save the file, I will rerun capture and reload .env automatically."

• If the capture flow reports a redirect to a login/authentication screen and the required login inputs were already provided, continue through the authenticated capture pre-step below instead of treating the redirect as a manual-only blocker.
• If the capture flow reports unexpected content (error page, blank page, different route): if the verifying agent has vscode/askQuestions, the immediate next action MUST be a vscode/askQuestions call such as: "The page at [URL] shows [description]. Is this the correct URL for [component name]?" Do not ask in plain assistant text first.
• If the capture flow cannot find the expected component on the confirmed page: if the verifying agent has vscode/askQuestions, raise the blocker through that tool immediately rather than a freeform reply.
• If any input is missing or any blocker is encountered, stop and resolve it through vscode/askQuestions when that tool is available to the verifying agent — do not proceed, do not fall back to code-level review, and do not skip the verification step

Authenticated capture pre-step

• Use this only when the user has explicitly authorized a genuine login and either populated the local .env contract derived from the real login form, completed the redirected real login form in the open browser session, or supplied an already-authenticated storage-state path.
• A genuine login means using the application's real sign-in UI exactly as an ordinary user would. It is allowed. Bypass is not.
• Standard local env contract: in the target repo .env file, set one env var per required login field using the derived naming rule TSH_UI_LOGIN_<NORMALIZED_FIELD_KEY>, where NORMALIZED_FIELD_KEY comes from name -> autocomplete -> id -> visible label text, normalized to uppercase snake case.
• Default path: if the redirect lands on a standard credential form, ask the user to populate the exact derived .env vars and confirm when the file is saved, then rerun capture with .env reloaded before filling the login form. Use direct in-browser login or a prepared storage-state path only for non-standard auth flows such as SSO, MFA, or captcha or when .env automation is not workable.
• Never ask the user to paste the password into chat. Read the env vars only at runtime and do not echo their values back into artifacts, reports, or tool output.
• Preferred pattern: perform the real login once, state-save to a secret path outside specifications/**, then state-load that path for each later capture iteration so the authenticated session is reused instead of recreated.
• Never write credentials into task specs, reports, artifacts, or committed files. Never seed cookies, tokens, localStorage, or sessionStorage by hand.

Step 2: Get EXPECTED from Figma — MANDATORY, runs BEFORE capture

This step is mandatory and always runs before capturing the implementation. A verification without fresh Figma EXPECTED data is INVALID. EXPECTED comes ONLY from the `figma` MCP tools — never open a figma.com URL (or any Figma link) in the Playwright/CLI browser to "fetch" the design, and never screenshot the Figma web app, its login page, or an error page as the reference. The browser is for the running app (ACTUAL) only. Do these in order:

1. Resolve the Figma node from the supplied Figma URL (extract fileKey + nodeId). If the URL or node cannot be resolved, raise it through vscode/askQuestions (when that tool is available), report VERIFICATION NOT RUN, and stop. Never continue without a resolved node.
2. Export the Figma node image via the `figma` MCP and SAVE it to the shared verification directory as specifications/<task-id>/ui-verification/figma-expected.png (the parent directory of the iteration directories defined in Step 3). Use the figma MCP's node-image / screenshot export — not a browser screenshot. This file is REQUIRED for the verification item and must be the real design export; it is the visual reference the comparison is judged against. Do not keep it only in memory or a tool response; it must exist on disk in the shared verification directory. If the figma MCP is not available in this workspace, that is a blocker: do NOT fall back to the browser and do NOT save any non-design image as figma-expected.png — report VERIFICATION NOT RUN and use vscode/askQuestions to ask the user to enable the Figma MCP or provide an exported reference image.
3. Extract the design specifications to compare against:

• Layer hierarchy and component structure
• Layout direction, alignment, spacing
• Frame width (use it as the capture viewport width in Step 3)
• Typography, colors, radii, shadows
• Component variants and states

ENSURE-OR-FETCH: At the start of every pass, check whether a valid shared figma-expected.png (a real design export) already exists at specifications/<task-id>/ui-verification/figma-expected.png for the current verification item. If it is missing, export it now via the figma MCP (steps 1–2 above). If it already exists and the Figma URL/node is unchanged, reuse it — do not re-export it for each iteration. Only after a genuine export failure (the figma MCP is unavailable, Figma cannot be reached, the node cannot be resolved, or the file cannot be written) do you report VERIFICATION NOT RUN, surface the blocker through vscode/askQuestions, and stop. Never browser-scrape Figma, never save a browser/login/error screenshot as figma-expected.png, and never proceed to compare against memory, source code, or the running app while a valid shared figma-expected.png is absent.

Step 3: Get ACTUAL from implementation

Use the tsh-ui-capture-worker capture flow to collect ACTUAL evidence from the running implementation. CLI capture is mechanical evidence collection only. The visual judge remains the reviewer brain comparing Figma EXPECTED against CLI ACTUAL using multimodal reasoning plus computed styles.

When the caller provides a Figma URL to tsh-ui-capture-worker, that worker may also export or ensure the shared figma-expected.png before opening the app page, purely as evidence preparation. This does not transfer design judgment from the reviewer; it only guarantees the EXPECTED artifact exists even when later ACTUAL capture is blocked by auth or page reachability.

The capture worker must use only the caller-provided full URL for the current pass. It never discovers its own URL, never replaces the caller-provided URL, never inspects project config to pick another port, and never launches or switches to another local app/server. If the delegated task does not include the confirmed full URL, treat that as a blocker and return immediately.

You MUST collect all three ACTUAL evidence types — a verification that skips any type is incomplete: