ajianaz/codomate/fullstack-qa-sop

Fullstack QA SOP v2

Overview

End-to-end QA standard operating procedure for fullstack web applications. Covers 5 test layers: functional, security, visual, responsive, and data integrity. Designed to be general-purpose — conditional checks skip gracefully when features don't apply.

Distilled from 150+ real tests across a production SaaS platform (85 functional + 67 security + 13 visual screenshots).

When to Use

• Before merging significant PR to develop/main
• After feature implementation (new pages, API endpoints, auth changes)
• Security audit requested for API backend
• Pre-deployment validation
• "Test everything" request

Pipeline Architecture

┌──────────────────────────────────────────────────────────────┐
│                    Fullstack QA Pipeline                      │
├────────────┬────────────┬──────────┬──────────┬──────────────┤
│ Layer 1    │ Layer 2    │ Layer 3  │ Layer 4  │ Layer 5      │
│ Functional │ Security   │ Visual   │ Responsive│ Data        │
│ E2E        │ Audit      │ Screens  │ Viewports │ Integrity   │
│            │            │          │          │              │
│ Playwright │ Python     │ Playwright+│ Playwright │ psycopg2  │
│ + Node     │ urllib     │ Vision AI│           │             │
├────────────┴────────────┴──────────┴──────────┴──────────────┤
│              Conditional Skipping (graceful)                   │
│         Skip if feature not present in app                    │
└──────────────────────────────────────────────────────────────┘

Layer 1: Functional E2E — references/layer1-functional.md

Goal: Verify every page renders and every user flow works end-to-end using Playwright. Covers token injection patterns, test matrix by role, shadcn migration pitfalls, Svelte 5 auth race conditions, and false positive classification.

Layer 2: Security Audit — references/layer2-security.md

Goal: Systematic API security testing via Python urllib — no browser needed. Covers API helper, 9 security test categories, JWT attack tests, RBAC matrix discovery, and conditional skip logic. See also references/security-checklist.md for the full checklist.

Layer 3: Visual Screenshots — references/layer3-visual.md

Goal: Capture every page across roles for visual regression. Covers viewport configurations, dark/light mode toggle, screenshot quality checks (duplicate/blank detection), and Discord delivery.

Layer 4: Responsive Testing — references/layer4-responsive.md

Goal: Verify layout adapts correctly across breakpoints. Covers sidebar behavior, table overflow, touch targets, DOM-based responsive QA script, and column hiding rules.

Layer 5: Data Integrity — references/layer5-data-integrity.md

Goal: Verify database state consistency after all tests. Covers before/after count verification, referential integrity, constraint validation, and migration count checks.

Prerequisites

Reporting

Console Summary Format

═══════════════════════════════════════════
  FULLSTACK QA RESULTS
═══════════════════════════════════════════
  Layer 1 — Functional:    80/85 passed (94%) [5 flaky]
  Layer 2 — Security:      38/55 passed (69%) [12 skipped]
  Layer 3 — Visual:        13 screenshots captured
  Layer 4 — Responsive:    3 viewports tested
  Layer 5 — Data Integrity: 10/10 checks passed
───────────────────────────────────────────
  TOTAL: 131/150 passed (87%)
  SKIPPED: 12 (conditional — not applicable)
  FLAKY: 5 (pass individually)
═══════════════════════════════════════════

Severity Classification

Discord Report

Send in multiple messages (2000 char limit):

1. Summary — overall pass rate, layer breakdown table
2. Security findings — severity table
3. Functional issues — per-issue with steps to reproduce
4. Screenshots — batched with MEDIA: paths (max ~5 per message)
5. Recommendations — prioritized fix list

Common Bug Patterns — references/common-bug-patterns.md

Frequently encountered bugs across projects: CORS logic inversion, UUID→500 errors, rate limiter false negatives, double-unwrap API responses, SQL COALESCE type mismatches, missing DB columns in Go structs, and Svelte 5 auth race conditions.

Common False Positives — references/false-positives.md

Patterns that frequently trigger false alarms in static analysis: fmt.Sprintf in SQL, Docker template placeholders, centralized API clients, test file type errors, Vite IPv6 proxy issues, and JSON:API response shape differences.

Execution Checklist

• [ ] Backend running + health check passing
• [ ] Frontend running + accessible
• [ ] Database seeded with test data
• [ ] Fresh JWT tokens for all test roles
• [ ] Layer 1: Functional E2E complete
• [ ] Layer 2: Security audit complete
• [ ] Layer 3: Visual screenshots captured
• [ ] Layer 4: Responsive breakpoints tested
• [ ] Layer 5: Data integrity verified (before/after counts)
• [ ] Report generated and sent
• [ ] All MEDIA: paths valid and accessible

See Also

• playwright-local-qa — SvelteKit Playwright patterns, token injection, mock API, shadcn responsive
• go-api-e2e-testing — Go API security testing, migration runner, RBAC matrix discovery
• dogfood — Exploratory browser QA, issue taxonomy, evidence collection
• systematic-debugging — Root cause analysis for bugs found during QA