<< All versions

Skill v1.0.0

currentLLM-judged scan90/100
yuga-hashimoto/open-test-pilot/review-tests
──Details
PublishedOctober 21, 2026 at 11:36 AM
Content Hashsha256:2b34deccaa7998ff...
Git SHAc6ad607cb68b
──Files
Files (1 file, 2.8 KB)
SKILL.md2.8 KBactive
SKILL.md · 23 lines · 2.8 KB

version: "1.0.0"


Review tests

Final check before proposing a Manifest (new or changed) to a human. Work through this checklist explicitly — treat every unchecked item as a reason to keep iterating, not a reason to note it and move on.

Checklist

  • Coverage vs. findings — does the Manifest actually exercise the route/form/endpoint that analyze-repository and design-tests identified, including the specific business outcome (not just "page loads")? Any coverage gap the selection table flagged as "write now" should be addressed or explicitly deferred with a reason.
  • Locator stability ranking — every selector/target should be at the top of the stability order from analyze-repository: data-testid/testTag/accessibilityIdentifier > role+accessible name / label text > static CSS/resource ID. Flag anything lower (generated class names, index-based selectors, deep CSS paths).
  • Assertion strength — assertions check the specific business-relevant outcome (exact text, specific status + schema), not a weak existence check where a stronger one is available. No assertion was silently removed or loosened since the last reviewed version without evidence recorded in repair-tests' trail.
  • Cleanup — every Step/setup that creates data, state, or a session has a matching cleanup entry so re-running the test doesn't accumulate junk or collide with other runs.
  • Secret handling — secrets: entries only, referenced as ${secret:NAME}; no literal credential, token, or PII anywhere in the Manifest, comments, or generated code.
  • Deterministic IDs — Manifest id, every Step id, and every Action id are stable kebab-case strings tied to business meaning, not counters; no duplicate IDs anywhere in the document.
  • Permissions and artifact policy — permissions.networkAccess/fileSystem match what the test actually needs (least privilege); artifacts.screenshots/traces are set to capture enough evidence for a future failure without being wasteful (e.g. don't turn on before-and-after screenshots for a pure API check).
  • Runner requirements — runner.minBrowsers (or mobile capabilities) match what the flow needs, nothing more.
  • Does this test hide a product defect? — re-check the most recent run's evidence: is any assertion tuned to match current (possibly broken) behavior rather than intended behavior? If evidence suggests the app itself is wrong, this belongs in analyze-failure's PRODUCT_DEFECT/SPECIFICATION_MISMATCH path — reported, not encoded as "expected."

Output

A pass/fail per checklist item with a one-line note, and an overall verdict: ready to propose (publish-tests) or send back for another design-tests/generate-manifest/repair-tests pass with the specific gaps listed.

All versions