kmshihab7878/claude-code-setup/test-gap-analyzer

# Python coverage
python -m pytest tests/ --cov=src/coremind --cov-report=json --cov-report=term-missing -q
 
# Frontend coverage
cd frontend && npx vitest run --coverage
 
# E2E coverage (count vs. critical user flows)
# Compare existing E2E tests against documented user flows

RISK SCORE = (Complexity × Change Frequency × User Impact) / Existing Coverage
 
Where:
  Complexity:       High (10) = many branches, external calls, state mutation
                    Medium (5) = moderate logic
                    Low (1) = simple getters, constants
 
  Change Frequency: Count of git commits touching this file in last 90 days
 
  User Impact:      Critical (10) = auth, payments, data mutation, API endpoints
                    High (7) = core business logic, agent execution
                    Medium (4) = internal utilities, helpers
                    Low (1) = dev tools, scripts, docs
 
  Existing Coverage: 0-100% (inverted — lower coverage = higher priority)

# Test Gap Report — [Date]
 
## Coverage Summary
| Suite | Current | Target | Delta | Status |
|-------|---------|--------|-------|--------|
| Python (pytest) | [X]% | 80% | [+/-Y]% | [OK/GAP] |
| Frontend (Vitest) | [X]% | 80% | [+/-Y]% | [OK/GAP] |
| E2E (Playwright) | [X/Y flows] | all critical | [N missing] | [OK/GAP] |
| API contracts | [X/Y endpoints] | all public | [N missing] | [OK/GAP] |
 
## Top 10 Gaps by Risk Score
 
| Rank | File/Function | Gap Type | Risk Score | Coverage | Recommendation |
|------|--------------|----------|------------|----------|---------------|
| 1 | [path:function] | [type] | [score] | [%] | [test type + approach] |
 
## Regression Debt
[List of production bugs without regression tests]
 
## Flaky Test Report
[List of flaky tests with failure rate and last occurrence]
 
## Improvement Trend
| Metric | Last Month | This Month | Delta |
|--------|-----------|------------|-------|
| Line coverage | [X]% | [Y]% | [+/-Z]% |
| Branch coverage | [X]% | [Y]% | [+/-Z]% |
| Bug escape rate | [X] | [Y] | [+/-Z] |
| Flaky test count | [X] | [Y] | [+/-Z] |

## Test Target: [File:Function]
 
**Gap type:** [Critical path / High-change / Branch / etc.]
**Risk score:** [X]
**Current coverage:** [X]%
**Why it matters:** [What could go wrong if this isn't tested]
 
### Tests to Write
1.`test_[function]_happy_path` — [expected behavior]
2.`test_[function]_returns_error_when_[condition]` — [error handling]
3.`test_[function]_handles_[edge_case]` — [boundary condition]
4.`test_[function]_with_[invalid_input]` — [input validation]
 
### Test Approach
-Pattern: [AAA / Property-based / Contract / E2E]
-Dependencies: [Real DB / Mock external / Fixture]
-Estimated effort: [S/M/L]

MEASURE:
  - Did new tests catch any existing bugs? (validation)
  - What's the coverage delta? (progress)
  - Any new flaky tests introduced? (quality)
  - Bug escape rate trend (effectiveness)
 
REINFORCE:
  - Test patterns that caught real bugs → prioritize more of these
  - Files that keep needing more tests → flag for refactoring
  - Coverage plateaus → switch from unit to property-based/mutation testing
 
REGENERATE:
  - Update gap report with new coverage data
  - Re-rank remaining gaps
  - Feed into next sprint's test targets

WEEKLY:
  Monday:  Run coverage scan + produce gap report
  Tuesday: Prioritize top 5 gaps for the sprint
  Sprint:  Write tests targeting gaps
  Friday:  Re-scan, measure improvement, feed into next week
 
MONTHLY:
  Trend analysis — are we improving?
  Flaky test cleanup sprint
  Regression debt review

Stage 1 (Intelligence):  Coverage scan → gap identification → risk ranking
Stage 2 (Context):       Load quality/domain.md + quality/rules.md before any test writing
Stage 3 (Generation):    Produce test targets → write tests in parallel
Stage 4 (Split):         Auto-generated unit tests (T0) vs. human-designed E2E (T1)
Stage 5 (Loop):          Measure coverage delta → reinforce patterns → regenerate targets