memorysaver/project-understanding/e2e-test

E2E Testing

Policy first. `policy.md` is the single source of truth for *which tiers gate a

layer, what environment to dogfood against (`none` / `local` / `deployed:<url>`), and when*

(pre-merge / post-deploy). /aep-build and /aep-wrap read it. If a tier or the journey library is

absent from this skill, policy.md says why.

Three tiers. Pick the one that matches the question being asked:

A green scripted gate is NOT proof the live product works — every layer gate has a manual journey half. Conversely, never debug machinery against a live environment — write a scripted test.

Tier 1 — Scripted gates (bun test)

Owned by the project's test framework. Run during /aep-build Phase 4 with the project's test command. These cover deterministic machinery; they are not the dogfood.

Tier 2 — Journey dogfood (declarative BDD, agent-executed)

The dogfood plan is a library of natural-language, BDD-style user-story journeys in `journeys/`, one per capability area, covering the shipped surface layer by layer. A journey describes intent (Given / When / Then / Verify), not a click script — the executing agent translates each step into the right tool calls.

Which tool drives the UI is resolved by [`tool-selection.md`](./tool-selection.md) — it picks the browser/device automation tool from the journey's target (web/mobile/desktop), the host, and any pinned preference, with health probes and graceful degrade. Journeys themselves stay tool-agnostic.

Execution protocol (how an agent runs a journey)

1. Pick a journey + scenario(s) from `journeys/`.
2. Clean slate — seed the _target_ environment. Run scripts/seed.sh against the same env the

journey runs on (from policy.md dogfood_target): local → bash scripts/seed.sh; deployed:<url> → SERVER_URL=<url> bash scripts/seed.sh (never seed local while dogfooding a deployed target — the journey would verify against stale/absent fixtures). Where the product has a real sign-in, auth is browser-simulated through the UI — not an API auto-login.

1. Read intent. You are given Given/When/Then — intent, not a recipe. Translate each step into tool

calls using the tool from tool-selection.md.

1. Drive + verify. Drive the UI with the resolved tool; assert backend state via the API. Ground

rules: fresh element refs each step, scroll off-screen elements into view, verify-state-before-claiming.

1. Record + report. Screenshots + API JSON as evidence; report PASS/FAIL per `Then`, not "looks

done". A Verify line that can't be confirmed is a FAIL.

1. Close coverage. Build the layer's coverage matrix — map each layer acceptance criterion to the

scenario Verify (or a Tier-1 case / Tier-3 API check) that proves it. For any uncovered criterion, author the missing scenario/case now and run it; loop until every criterion is covered (or a WAIVER: <reason> is recorded). One green journey is not coverage.

Layer gate (two-phase, covered)

A layer gate is two-phase, and green means covered — not "one journey passed":

• `scripted_passed` — the layer's Tier-1 framework tests are green (machinery proven).
• `passed` — scripted_passed + the applicable journey / API tiers green + every layer

acceptance criterion proven (coverage.criteria_covered == criteria_total) + prior-layer journeys replay green (regression).

Run the matching journey, close any coverage gap (step 6), and record evidence in docs/layer-gates/<layer>.md — the two matrices (acceptance-traceability + scripted-coverage), the checklist, and any WAIVER: lines. Copy `layer-gate-evidence.template.md` per layer. /aep-wrap then performs the two-phase flip and asks the human before advancing. Bugs found route through /aep-reflect (bug → high-priority story; "works but feels wrong" → calibration). The full loop is in the aep-e2e-skill-scaffolding skill's references/layer-gate-loop.md.

Tier 3 — API drivers

For backend verification and things browsers are slow at (async/streamed turns, bulk setup, state-tree diffs). Write .dev-workflow/dogfood-<feature>.{mjs,…}: sign in → call the API → assert. These are gitignored throwaways — reusable patterns belong in this SKILL.md, not the scripts.

Setup

Source ports before running any test:

source .dev-workflow/ports.env    # BASE_URL=http://localhost:3001  SERVER_URL=http://localhost:3000

Account reset & seed

scripts/seed.sh converges the test account/database to a deterministic fixture via the public API (idempotent — safe to re-run). Run it before scenario dogfoods; stale or empty data hides bugs.

bash scripts/seed.sh                                  # local (reads .dev-workflow/ports.env)
SERVER_URL=<prod-url> [SECRET=value …] bash scripts/seed.sh   # target a deployed env (prefix secret VAR=value assignments)

Keeping this current

When a feature ships or changes behavior, update the matching journey in `journeys/` in the same PR. A stale journey silently shrinks regression coverage. New capability → new journey (copy the template in `journeys/README.md`).