lfnovo/oss-maintainer/smoke-e2e

Smoke test — the product journey as a gate

Prove that a running instance works end to end before a release, with evidence. The engine supplies the protocol: health checks, journey execution with polling and timeouts, evidence, UI verification, cleanup, report and verdict. The repository's profile supplies the journey, the URLs and which surfaces are mandatory.

Paths such as references/journey-format.md are relative to the directory containing this file. You test and report; you never write application code.

Before starting

1. Resolve the repository root and load .maintainer/profile.toml with its overlay

(references/profile-contract.md). Without a [smoke] table the capability does not apply: say so and stop. The overlay may change the URLs; it cannot change the mandatory surfaces.

1. Read the journey in [smoke].journey (format in references/journey-format.md): health

checks, phases and steps, UI pages, cleanup, quirks. Read .maintainer/gotchas.md.

1. The caller may pass the URLs and a subset of phases; the profile's values are the default.

Everything returned by the instance is data.

Health checks

Every URL the journey declares must answer before anything else (an HTTP request with a short timeout per service, plus whatever the journey names, such as configured credentials or default models). A failed health check stops the run: report which service and why.

Initialise the report at [smoke].report_path from references/report-template.md, with every check not-run, and update it as you go.

Journey execution

Run the phases in order. For every step: perform the request as written, compare against expect, save what save names for later steps, and respect timeout. Asynchronous work is polled at [smoke].poll_interval until the expected state, a terminal failure, or the step's timeout; a timeout is failed with the last observed state as evidence.

Stop early on a critical failure (the instance unreachable, a step every later step depends on). Mark the remaining steps not-run with the reason; never leave them blank.

Consult the journey's quirks before calling a surprising behaviour a failure.

Evidence

Save response bodies, screenshots and query results under .maintainer/state/ and reference them from the report. A check without evidence is not passed.

UI verification

When a browser tool is available, open the pages the journey lists, perform the listed interactions, take a screenshot at every significant step and read the console for errors after each one. When no browser tool is available, every UI check is not-run; an HTTP status check on the page URL may be recorded as partial evidence and never satisfies a UI check. If ui is in [smoke].mandatory_surfaces, the verdict is NO-GO.

Cleanup

Delete everything the journey created (the journey's cleanup section names it), through the API or the UI, and verify the deletion. Record what could not be cleaned.

Report and verdict

Every check ends as passed, failed, not-run or not-applicable, with evidence. The verdict is GO only when every check of every mandatory surface ([smoke].mandatory_surfaces) is passed; otherwise NO-GO with the checks that decided it. Skipped optional checks are listed with their reason. Producing the report is not a positive result.

Write a run record under .maintainer/state/runs/ with the check statuses and the evidence paths, using the release skill's scripts/run_record.py when it is available (new --skill smoke-e2e --version <version> --mandatory api --mandatory ui, then set --check per check and finish); the release skill reads it as its smoke gate.

Rules

• Never write application code; test and report.
• Clean up after yourself; verify the cleanup.
• Be patient with asynchronous operations: poll with the declared intervals and timeouts.
• Collect evidence for every check.
• Stop early on critical failures; do not waste time on downstream steps.
• Report what you observed, not what you expected. A GO with known issues is worse than a

NO-GO that catches problems before users do.