version: "1.0.1" name: qa-testing-strategy description: Risk-based test strategy for software delivery. Use when defining coverage, setting CI gates, managing flaky tests, or establishing release criteria.

QA Testing Strategy (Jan 2026)

Risk-based quality engineering strategy for modern software delivery.

Core references: curated links in data/sources.json (SLOs/error budgets, contracts, E2E, OpenTelemetry). Start with references/operational-playbook.md for a compact, navigable overview.

Scope

• Create or update a risk-based test strategy (what to test, where, and why)
• Define quality gates and release criteria (merge vs deploy)
• Select the smallest effective layer (unit → integration → contract → E2E)
• Make failures diagnosable (artifacts, logs/traces, ownership)
• Operationalize reliability (flake SLO, quarantines, suite budgets)

Use Instead

Quick Reference

Default Workflow

1. Clarify scope and risk: critical journeys, failure modes, and non-functional risks (latency, data loss, auth).
2. Define quality signals: SLOs/error budgets, contract/schema checks, and what blocks merge vs blocks deploy.
3. Choose the smallest effective layer (unit → integration → contract → E2E).
4. Make failures diagnosable: artifacts + correlation IDs (logs/traces/screenshots), clear ownership, deflake runbook.
5. Operationalize: flake SLO, quarantine with expiry, suite budgets (PR gate vs scheduled), dashboards.

Test Pyramid

           /\
          /E2E\          5-10% - Critical journeys
         /------\
        /Integr. \       15-25% - API, DB, queues
       /----------\
      /Component \       20-30% - UI modules
     /------------\
    /   Unit      \      40-60% - Logic and invariants
   /--------------\

Decision Tree: Test Strategy

Need to test: [Feature Type]
    │
    ├─ Pure business logic/invariants? → Unit tests (mock boundaries)
    │
    ├─ UI component/state transitions? → Component tests
    │   └─ Cross-page user journey? → E2E tests
    │
    ├─ API Endpoint?
    │   ├─ Single service boundary? → Integration tests (real DB/deps)
    │   └─ Cross-service compatibility? → Contract tests (schema/versioning)
    │
    ├─ Event-driven/API schema evolution? → Contract + backward-compat tests
    │
    └─ Performance-critical? → k6 load testing

Core QA Principles

Definition of Done

• Strategy is risk-based: critical journeys + failure modes explicit
• Test portfolio is layered: fast checks catch most defects
• CI is economical: fast pre-merge gates, heavy suites scheduled
• Failures are diagnosable: actionable artifacts (logs/trace/screenshots)
• Flakes managed with SLO and deflake runbook

Shift-Left Gates (Pre-Merge)

• Contracts: OpenAPI/AsyncAPI/JSON Schema validation
• Static checks: lint, typecheck, secret scanning
• Fast tests: unit + key integration (avoid full E2E as PR gate)

Shift-Right (Post-Deploy)

• Synthetic checks for critical paths (monitoring-as-tests)
• Canary analysis: compare SLO signals and key metrics before ramping
• Feature flags for safe rollouts and fast rollback
• Convert incidents into regression tests (prefer lower layers first)

CI Economics

Flake Management

• Define: test fails without product change, passes on rerun
• Track weekly: flaky_failures / total_test_executions (where flaky_failure = fail_then_pass_on_rerun)
• SLO: Suite flake rate ≤ 1% weekly
• Quarantine policy with owner and expiry
• Use the deflake runbook: template-flaky-test-triage-deflake-runbook.md

Common Patterns

AAA Pattern

it('should apply discount', () => {
  // Arrange
  const order = { total: 150 };
  // Act
  const result = calculateDiscount(order);
  // Assert
  expect(result.discount).toBe(15);
});

Page Object Model (E2E)

class LoginPage {
  async login(email: string, password: string) {
    await this.page.fill('[data-testid="email"]', email);
    await this.page.fill('[data-testid="password"]', password);
    await this.page.click('[data-testid="submit"]');
  }
}

Anti-Patterns

Do / Avoid

Do

• Write tests against stable contracts and user-visible behavior
• Treat flaky tests as P1 reliability work
• Make "how to debug this failure" part of every suite

Avoid

• "Everything E2E" as default
• Sleeps/time-based waits (use event-based)
• Coverage % as primary quality KPI

Feature Matrix vs Test Matrix Gate (Release Blocking)

Before release, run a coverage audit that maps product features/backlog IDs to direct test evidence.

Gate Rules

• Every release-scoped feature must map to at least one direct automated test, or an explicit waiver with owner/date.
• Evidence must include file path and test identifier (suite/spec/case).
• "Covered indirectly" is not accepted without written rationale and risk acknowledgment.
• If critical features have no direct evidence, release is blocked.

Minimal Audit Output

• feature/backlog id
• coverage status (direct, indirect, none)
• evidence reference
• risk level
• owner and due date for gaps

Resources

Templates

Data

Related Skills

• qa-debugging — Debugging failing tests
• qa-agent-testing — Testing AI agents
• software-backend — API patterns to test
• ops-devops-platform — CI/CD pipelines

Ops Gate: Release-Safe Verification Sequence

Use this sequence for feature branches that touch user flows, pricing, localization, or analytics.

# 1) Static checks
npm run lint
npm run typecheck
 
# 2) Fast correctness
npm run test:unit
 
# 3) Critical path checks
npm run test:e2e -- --grep "@critical"
 
# 4) Instrumentation gate (if configured)
npm run test:analytics-gate
 
# 5) Production build
npm run build

If a Gate Fails

1. Capture exact failing command and first error line.
2. Classify: environment issue, baseline known failure, or regression.
3. Re-run only the failed gate once after fix.
4. Do not continue to later gates while earlier required gates are red.

Agent Output Contract for QA Handoff

Always report:

• commands run,
• pass/fail per gate,
• whether failures are pre-existing or introduced,
• next blocking action.

Fact-Checking

• Use web search/web fetch to verify current external facts, versions, pricing, deadlines, regulations, or platform behavior before final answers.
• Prefer primary sources; report source links and dates for volatile information.
• If web access is unavailable, state the limitation and mark guidance as unverified.