kunolu/640-skills/project-validation

Project Validation Skill

Use this Skill after code changes.

General Rules

• Prefer commands already defined by the project.
• When rtk is available, prefer rtk for non-reporting commands; for unit tests, API / integration tests, Playwright Web E2E, Maestro Mobile / Hybrid E2E, or any command that must generate report files, first apply the rtk and reporting-test Gate.
• Do not bypass project configuration.
• Do not modify lock files unless required by the task.
• If full checks are expensive, run focused checks first.
• State which checks were skipped and the remaining risks.

rtk and Reporting-Test Gate

rtk is a command-output compression layer, not a test runner. Before executing validation commands, first distinguish between “only terminal facts are needed” and “file side effects must be produced.”

• lint, typecheck, static analysis, build, read-only checks, or diagnostic commands that do not depend on persisted reports can generally prefer rtk.
• If unit tests, API / integration tests, Playwright Web E2E, Maestro Mobile / Hybrid E2E, Flutter / Xcode / Gradle / Maven, or similar test commands need to retain coverage, JUnit, HTML, JSON, trace, raw report, or Markdown summaries in the current run, use the project-native command by default, or a no-cache / report-safe command explicitly provided by the project.
• Wrap a reporting-test command with rtk only after confirming that rtk will not produce a cache hit, replay output, or skip the runner’s file-writing side effects for that command, and that the report path can be verified.
• If a reporting test has already been executed with rtk, verify that the expected report files exist, that their mtime / size changed after the current run, and that their contents correspond to the current command and case / spec / flow. If a file is missing, stale, empty, has mismatched contents, or the output shows a cache hit / replay / skipped write, immediately rerun with the native command and treat the native-command result and persisted reports as authoritative.
• stdout results from custom API scripts, Playwright, Maestro, or unit runners cannot replace the report-file gate; when such a command is part of formal validation, it must persist a raw report / native report and a same-stem Markdown summary, or be marked blocked.
• The final output must report rtk: used / skipped-for-report / fallback-native / not-available / not-needed, and explain the reason.

Book-Derived Validation Supplement

Project validation is responsible for selecting and running commands such as lint / test / build / typecheck; it does not replace book-derived skills.

For production-path-related changes to services, APIs, background jobs, queues, external integrations, data pipelines, or deployment-sensitive components, after basic project validation you must proactively determine whether to invoke book-release-readiness. If validation exposes risks involving data consistency, migration, replay, idempotency, or cross-service data flows, return to book-ddia-data-design to complete the design / check conclusions before finishing.

This step records only the current task’s risks, validation gaps, and remaining risks; do not add task-unrelated refactoring or test frameworks because of the production risk review.

BDD / Gherkin Validation Supplement

When the task adds or modifies user-visible behavior, or the diff contains .feature / persistent BDD specification paths, BDD consistency must be validated.

Return scenario authoring, review, or backfilling issues to the gherkin-bdd Skill; this Skill is responsible only for selecting and executing post-change validation and reporting risks.

Check in this order:

1. Confirm whether the user-visible behavior has a corresponding persistent BDD scenario; when skipping pure internal changes or semantically neutral UI polish, record the reason for skipping.
2. Check whether the language decision for .feature / persistent BDD specifications has been followed and is consistent with the file contents:

• If the project already has .feature files, additions or modifications must follow the existing Gherkin language and keyword style of the same bounded context or functional area.
• If the project previously had no .feature files and the user has not explicitly requested another language, scenario titles, descriptions, and step text in new .feature files must be in Chinese by default; use English for Gherkin structural keywords.
• English product names, code identifiers, and domain-specific names may remain in English, but an entire new .feature file must not be written in English.
• Do not rely only on git diff --check to determine language correctness; manually review it, or use lightweight checks to help identify obvious violations.

1. If the project previously had no .feature files, and a new .feature file contains no Chinese characters outside comments, tags, tables, doc strings, and structural keywords, with no user override or project-rule override, mark BDD as blocked and return to gherkin-bdd to correct the language first.
2. If the project already has a Gherkin runner, such as Cucumber, behave, pytest-bdd, or cucumber-js, or has a BDD command in a package / Makefile / CI configuration, prefer the project-defined BDD command.
3. If there is no Gherkin runner, do not proactively introduce a new framework; use the project’s existing test framework to run unit / integration / E2E tests traceable to the scenarios.
4. Confirm that every added or modified scenario is traceable to an automated test. Traceability may use test names, comments, directory structure, or project conventions.
5. Scenarios that cannot be automated must have @todo or a project-equivalent marker, the blocking reason, and temporary manual-validation instructions.
6. If the PRD, .feature, tests, and code conflict, return to specification alignment first; do not use validation results to conceal the conflict.
7. For separate frontend and backend repositories, cross-service flows, Web + API, Mobile + API, or Hybrid flows, check whether context completeness has been recorded as Cross-repo context: complete / contract-only / environment-only / missing.
8. When mocks are needed, confirm that mock behavior comes from an API contract, schema, real response sample, existing fixture, or user confirmation; otherwise mark Mock Strategy as blocked and do not generate tests using guessed mocks.

The final output must state:

• BDD: run / traceable / blocked / skipped.
• The affected .feature or persistent BDD specification paths.
• BDD language status: following the project’s existing style, default Chinese scenario text + English keywords, explicit user override, or the reason for blocked.
• The BDD runner or traceability-test command that was run.
• Cross-repo context: complete / contract-only / environment-only / missing; state not-needed if not relevant.
• API Contract: verified / user-provided / stale / missing / not-needed.
• Mock Strategy: none / contract-backed / user-approved / blocked.
• Unautomated scenarios, blocking reasons, and remaining risks.

Validation Evidence Sources and Version Contract

Apply references/validation-evidence-contract.md only when formal test reports must serve as PR evidence or be read by the knowledge base. Ordinary local diagnostics, unpublished debugging reports, and one-off troubleshooting do not require an evidence sidecar.

When the project has a configured product registry, invoke the decision entry point of knowledge-base-integration to determine the Evidence Contract, Intent, and Targets; do not infer the purpose from report directories, branch names, or arbitrary CI environments. Knowledge-server smoke is generated by that Skill’s smoke entry point as a P1.1 bundle; this Skill remains responsible for project-native validation and the report-quality Gate. P1.1 accepts only runner reports newly created or refreshed after the current command began and same-stem Chinese Markdown; the aggregate envelope must reference the artifact manifest, checksums, and actual runner attestation. Evidence must not be marked as passed when it is stale, missing, not in Chinese, has inconsistent stems, has inconsistent report checksums / digests, fails schema / semantic validation, or comes from an untrusted environment.

• Developer-local evidence uses Evidence Source: developer-local; evidence generated by a CI runner uses Evidence Source: ci; independent revalidation by the knowledge-base server uses Evidence Source: knowledge-server. These three must not overwrite or impersonate one another.
• Formal evidence must record the repository key, original source ref, full commit SHA, worktree state, trigger, and creation time. branch_slug is used only for filenames and is not a code-version identifier.
• A dirty worktree may only be marked Source Revision: dirty and Evidence Publication: local-only; it cannot serve as formal proof of the PR head.
• PR evidence must exactly match the current PR head SHA; old evidence automatically becomes invalid after a new commit is added.
• Results produced before the Phase 3.4 commit plan or before creating a commit may only be recorded as local evidence status. After committing, and before publishing or updating the PR Check, evidence must be regenerated or revalidated against the final PR head SHA, and the same-stem sidecar / aggregate envelope must be updated; this applies to both developer-local and ci.
• ci evidence must come from a clean checkout and use Source Revision: exact. CI execution does not mean the evidence has been published: mark it published only after the target system receives it, not-configured when no publisher is configured, and blocked when publication is required but fails; CI evidence must not be marked local-only.
• Knowledge-base-server evidence must record the exact revision set of all participating repositories and record Environment Alignment. When a target branch such as staging is specified, resolve it to an exact SHA before running.
• .feature files are not required to add Feature IDs or Scenario IDs. Identify behavior sources using the repository key, feature path, Feature / Rule / Scenario name, optional Examples fingerprint, source ref, and SHA.
• Generic or historical report evidence continues to use references/validation-evidence.schema.json (schemaVersion: 1). v1 featureSources[] plus reports[] co-membership is not BDD traceability. The v1 Schema only checks digest shape; producers and consumers must recompute each referenced report SHA-256 against file bytes and reject mismatches.
• Scenario-backed execution evidence must use references/validation-evidence.v2.schema.json and scripts/validate_validation_evidence.py. The validator must parse SHA-verified junit-xml-v1 or playwright-json-v1 bytes and require the matched passed case to carry sbtd.sourceLocatorDigest equal to the recomputed locator. HTML/TXT/arbitrary JSON and sidecar labels alone fail closed.
• Each formal report may generate a same-report-stem .evidence.json in the same directory; a cross-tool orchestrator may also generate one aggregate envelope in an isolated runtime / evidence bundle. Every envelope must pass the matching Schema, and v2 envelopes must also pass the semantic validator.
• Evidence must be sanitized before publication. published only means that the target system has received the evidence; it does not mean the tests passed, and smoke-only, contract-backed, mock-backed, or app-mocked must not be promoted to full-stack.

For relevant tasks, additionally report in the final output: