howells/arc/testing

<tool_restrictions> Ask one question at a time. In Claude Code use AskUserQuestion; elsewhere ask a single concise plain-text question. Keep any lead-in to 2-3 sentences. Don't narrate missing tools or fallbacks.

EnterPlanMode and ExitPlanMode are banned. This skill is Arc's own structured testing process. </tool_restrictions>

<arc_runtime> Requires the full Arc bundle. Arc-owned paths (agents/, references/, disciplines/, templates/, scripts/, rules/, skills/) resolve from the plugin root — the directory containing agents/ and skills/. Everything else is the user's repository. </arc_runtime>

Characterization Testing Workflow

Backfill focused tests around existing code before a risky change. The goal is not "more tests" in the abstract; it is a trustworthy safety net around behavior that must survive a refactor, migration, or bug fix.

Use this skill when:

• Existing code has little or no test coverage.
• A refactor needs a behavior-preserving safety net first.
• A god file, duplicated implementation, or tangled module needs characterization before decomposition.
• A performance optimization needs current behavior pinned before changing data structures, batching, memoization, caching, or ordering.
• A bug fix touches unclear behavior and you need to capture the current contract before changing it.
• Coverage reports show gaps around important public behavior.
• Auth, API, state, or browser flows need targeted tests before launch or audit remediation.

Do not use this skill as the normal new-feature workflow. For new work, use /arc:implement or a dedicated TDD skill so RED/GREEN/REFACTOR remains the governing loop.

<required_reading> Read before testing:

1. references/testing-patterns.md — Test philosophy, vitest/playwright patterns
2. references/testing-anti-patterns.md — What weak or misleading tests look like
3. rules/testing.md — Arc testing conventions, when the project has no rules of its own (see <rules_context>)
4. disciplines/change-impact-testing.md — loaded for its blast-radius framing only. It is a post-change discipline, so its revert-the-change sensitivity proof does not apply here; on a backfill run there is no change to revert, and the perturbation method in Step 4 is the sensitivity proof.
5. references/llm-api-testing.md — If testing LLM integrations
6. references/maintainability-review.md — If tests are being added before decomposing a god file or tangled module
7. references/complexity-optimization.md — If tests are being added before optimizing algorithmic complexity, rendering churn, or N+1 behavior

</required_reading>

Agents

Use specialist agents only when the slice is large enough to justify delegation:

<rules_context> Check for project testing rules:

Use Glob tool: .ruler/testing.md

• If `.ruler/` exists: Read rules from .ruler/, for MUST/SHOULD/NEVER constraints.
• If `.ruler/` doesn't exist: Read rules from rules/.

Detected project tooling wins over a rule that names a different runner: if the repo runs jest, back the safety net with jest rather than converting the suite to satisfy the rule.

Detect test framework and package manager: use the detection tables in references/testing-patterns.md. Fall back to package.json scripts for project-specific test commands.

</rules_context>

Process

Step 1: Confirm The Safety-Net Target

Ask one question only if the target is unclear:

AskUserQuestion:
  question: "What existing code or behavior needs a safety net before we change it?"
  header: "Test Target"
  options:
    - label: "A specific file/module"
      description: "Point me at the file, component, or module you're about to change"
    - label: "A route or API surface"
      description: "An endpoint, page, or public interface whose behavior must survive the change"
    - label: "A user flow"
      description: "An end-to-end journey (auth, checkout, signup) that needs pinning before a refactor"
    - label: "Something else"
      description: "Describe the behavior or blast radius you want protected"

Then identify:

• The files, routes, packages, components, or commands involved.
• The planned change or refactor the tests must protect.
• The public interfaces where behavior is observable.
• Any business-critical, auth, persistence, payment, data, or browser-flow risk.
• Any ordering, duplication, identity, mutability, pagination, permission, cache invalidation, or tenant/filtering behavior that an optimization must preserve.

Step 2: Establish The Baseline

Gather evidence before writing tests:

• Read the target code and nearby tests.
• Read recent commits or plans when they explain the intended behavior.
• Run the smallest existing relevant test command.
• If no test command exists, identify the project’s likely framework and package manager.
• Note current failures separately from new failures.

Do not silently fix production behavior during baseline work. If you discover an obvious bug, capture it as either:

• A current-behavior characterization test if the change is meant to preserve it.
• A failing desired-behavior test if the user is asking for the bug to be fixed.

Step 3: Map Public Behavior

List behavior in terms of callers or users, not internal implementation details.

Record it using the Safety Net structure in templates/safety-net.md.

Step 4: Add Tests One Vertical Slice At A Time

Confirm the chosen seam with the user before writing any test against it. Agreeing the boundary up front keeps testing effort on the target the user actually cares about.

For each slice:

1. Choose one public behavior.
2. Choose the smallest useful test level.
3. Write the test.
4. Run only the relevant test.
5. Prove the test is sensitive:

• For current-behavior characterization, the test may pass immediately. Temporarily perturb the assertion, fixture, or input to prove it fails for the right reason, then restore it.
• For desired behavior or bug fixes, follow RED/GREEN/REFACTOR. Do not change production code before the failing test exists.

1. Commit no temporary mutations.
2. Move to the next slice only after the current slice is trustworthy.

Step 5: Keep Test Seams Small

If existing code is hard to test:

• Prefer testing through an existing public interface.
• Extract only the smallest seam needed to observe behavior.
• Preserve behavior while extracting.
• Avoid large refactors before the safety net exists.
• Avoid mocking internal modules just to force a unit test.

Mocks are acceptable for true boundaries: network, time, filesystem, database, auth providers, payment providers, and external LLM APIs. Prefer real code inside the project boundary.

Step 6: Run Scoped Then Broader Verification

Run checks in widening order:

1. The single new test file or test name.
2. The relevant package or feature test suite.
3. The project’s normal test command.
4. E2E only when the risk is browser-level or cross-system.

When E2E output is verbose or flaky, dispatch e2e-runner with the exact test file and failure evidence.

Step 7: Report The Safety Net

End with a concise report.

Report using the Safety Net Result structure in templates/safety-net.md.

Mastra Agent Surfaces

If a Mastra agent surface is detected (packages/mastra, or @mastra/* in package.json), the useful safety net is often an eval/golden-set, not a conventional characterization test. Offer to load references/agent-evals.md and backfill an eval/golden-set that pins current agent behavior (expected outputs, tool-call shape, scorers) before the agent is changed.

Choosing Test Level

Coverage Guidelines

Auth Testing Quick Reference

Use this only when auth behavior is part of the safety net.

Clerk Testing

Integration tests:

• Mock useAuth and useUser hooks.
• Test loading, signed-in, and signed-out states.
• Mock getToken for API calls.

E2E tests:

• Create tests/auth.setup.ts for login flow.
• Store session in playwright/.auth/user.json.
• Use storageState in playwright.config.ts.

Common issues:

• Trying to mock ClerkProvider instead of hooks.
• Missing the isLoaded: false state.
• Hardcoding tokens instead of using a getToken mock.

WorkOS Testing

Integration tests:

• Mock getUser from @workos-inc/authkit-nextjs.
• Test with full user object including organizationId, role, and permissions.
• Test SSO redirect behavior.

E2E tests:

• SSO flows are slow; consider a test bypass endpoint.
• Create /api/auth/test-login for faster auth in test environments only.
• Store session state after auth.

Common issues:

• Missing organizationId in org-level features.
• Not testing permission checks.
• SSO redirect timing issues without proper waits.

Bypass Auth For Speed

For faster E2E tests, create a test-only auth endpoint: