<< All versions
Skill v1.0.0
currentLLM-judged scan90/100yuga-hashimoto/open-test-pilot/review-tests
──Details
PublishedOctober 21, 2026 at 11:36 AM
Content Hashsha256:2b34deccaa7998ff...
Git SHAc6ad607cb68b
──Files
Files (1 file, 2.8 KB)
SKILL.md2.8 KBactive
SKILL.md · 23 lines · 2.8 KB
version: "1.0.0"
Review tests
Final check before proposing a Manifest (new or changed) to a human. Work through this checklist explicitly — treat every unchecked item as a reason to keep iterating, not a reason to note it and move on.
Checklist
- Coverage vs. findings — does the Manifest actually exercise the route/form/endpoint that
analyze-repositoryanddesign-testsidentified, including the specific business outcome (not just "page loads")? Any coverage gap the selection table flagged as "write now" should be addressed or explicitly deferred with a reason. - Locator stability ranking — every
selector/targetshould be at the top of the stability order fromanalyze-repository:data-testid/testTag/accessibilityIdentifier> role+accessible name / label text > static CSS/resource ID. Flag anything lower (generated class names, index-based selectors, deep CSS paths). - Assertion strength — assertions check the specific business-relevant outcome (exact text, specific status + schema), not a weak existence check where a stronger one is available. No assertion was silently removed or loosened since the last reviewed version without evidence recorded in
repair-tests' trail. - Cleanup — every Step/setup that creates data, state, or a session has a matching
cleanupentry so re-running the test doesn't accumulate junk or collide with other runs. - Secret handling —
secrets:entries only, referenced as${secret:NAME}; no literal credential, token, or PII anywhere in the Manifest, comments, or generated code. - Deterministic IDs — Manifest
id, every Stepid, and every Actionidare stable kebab-case strings tied to business meaning, not counters; no duplicate IDs anywhere in the document. - Permissions and artifact policy —
permissions.networkAccess/fileSystemmatch what the test actually needs (least privilege);artifacts.screenshots/tracesare set to capture enough evidence for a future failure without being wasteful (e.g. don't turn onbefore-and-afterscreenshots for a pure API check). - Runner requirements —
runner.minBrowsers(or mobilecapabilities) match what the flow needs, nothing more. - Does this test hide a product defect? — re-check the most recent run's evidence: is any assertion tuned to match current (possibly broken) behavior rather than intended behavior? If evidence suggests the app itself is wrong, this belongs in
analyze-failure'sPRODUCT_DEFECT/SPECIFICATION_MISMATCHpath — reported, not encoded as "expected."
Output
A pass/fail per checklist item with a one-line note, and an overall verdict: ready to propose (publish-tests) or send back for another design-tests/generate-manifest/repair-tests pass with the specific gaps listed.