Ralph Review
Post-build verification and remediation skill. Run this after Ralph finishes a batch of iterations (or an entire service build) to catch issues before they compound.
Overview
Three-stage process:
- Verify — parallel agents audit what Ralph produced
- Cross-examine — agents see each other's findings and react
- Remediate — fix HIGH findings, verify fixes with trailing agents
Input Detection
Parse the user's input for:
- Service name or phase: which service/phase Ralph was working on (e.g.,
rideshare, tesla, dashboard, or phase-1) - Iteration range: e.g., "10-20" means audit iterations 10 through 20 (optional — if omitted, audit everything since last review)
Resolve Ralph artifact paths by searching:
- First try:
mock-apis/{service}/ralph/prd/prd-mock-{service}.json - Then try:
{service}/ralph/prd/*.json - Then try:
Plan/phase-{N}-*/ralph/prd/prd-phase-{N}.json - Fallback: glob for
**/ralph/prd/prd*.json and let user pick
Stage 0: Build Project Context
Before launching any agents, you MUST build a context block from the actual codebase.
Read these files (in order, skip if missing):
AGENTS.md — canonical patterns, conventions, tech stackCLAUDE.md — project instructions, dev commandspackage.json (root) — workspace structure, scripts, dependencies- The PRD file for this service/phase — stories, acceptance criteria
From these, construct a {context block} that includes:
- Tech stack (framework, language, package manager, monorepo tool if any)
- Project structure (apps, packages, key directories)
- Auth/security patterns (if documented in AGENTS.md)
- Key conventions (import patterns, naming, file organization)
- Dev commands (typecheck, lint, build, test)
- Quality gate commands (what to run and what "pass" looks like)
- Commit message format (from AGENTS.md if documented)
This context block gets injected into every agent prompt below where {context block} appears.
Stage 1: Quick Gates (Sequential)
Run these first. If they fail, fix before proceeding to deeper audit.
Detect the correct commands from package.json scripts or CLAUDE.md:
Gate 1: Typecheck
bash
# Use whichever exists: pnpm typecheck, npm run typecheck, npx tsc --noEmit
{TYPECHECK_COMMAND} 2>&1 | tail -30
If FAIL -> fix immediately before proceeding.
Gate 2: Tests
bash
# Use whichever exists: pnpm test, npm test, npx vitest run
{TEST_COMMAND} 2>&1 | tail -50
If FAIL -> fix immediately before proceeding. Tests are a hard gate — they must pass.
Gate 3: Lint (if available)
bash
# Use whichever exists: pnpm lint, npm run lint, npx eslint .
{LINT_COMMAND} 2>&1 | tail -30
If FAIL -> fix immediately before proceeding. If no lint script exists, skip.
Gate 4: PRD Status
bash
node -e "
const prd = require('{PRD_FILE_PATH}');
const stories = prd.stories;
const complete = stories.filter(s => s.status === 'complete');
const incomplete = stories.filter(s => !['complete','deferred'].includes(s.status));
console.log('Complete: ' + complete.length + '/' + stories.length);
console.log('Remaining: ' + incomplete.length);
incomplete.slice(0, 10).forEach(s => console.log(' - ' + s.id + ': ' + s.title));
"
Report gate results to user before proceeding:
Quick Gates:
Typecheck: {PASS/FAIL}
Tests: {PASS/FAIL} ({X passed, Y failed})
Lint: {PASS/FAIL/SKIPPED}
PRD: {X/Y} complete, {Z} remaining
Stage 2: Parallel Audit Agents (6 agents, all background)
Launch ALL 6 agents simultaneously using run_in_background: true.
- All agents:
model: "sonnet" — use Sonnet for all agents, never Haiku - All agents use
subagent_type: "Explore"
CRITICAL: Every agent prompt MUST include the context block built in Stage 0.
CRITICAL: Every agent prompt MUST include the Agent Roster — a list of all 6 agents and their focus areas. This lets each agent flag cross-cutting concerns for other agents to verify in Stage 2.5.
Include this in every agent prompt:
## Agent Roster (you are Agent {N})
1. Security Auditor — auth, access control, injection, secrets, HMAC
2. Architecture Auditor — patterns, conventions, file organization, dependencies
3. Data Layer Auditor — schema, queries, indexes, migrations, tenant isolation
4. UI/UX Auditor — pages, components, accessibility, loading/error/empty states
5. Integration Auditor — cross-package wiring, type flow, env vars, build pipeline
6. Gotcha Hunter — TODOs, hardcoded values, race conditions, missing awaits, console.log
If you find something outside your domain, note it as:
CROSS-REF [Agent N]: [what they should check]
Agent 1: Security Auditor (Model: sonnet)
{context block}
{agent roster}
## Your Focus: Security Audit
Audit all code produced by Ralph in this service/phase for security issues:
1. If an RBAC matrix or role-based access control exists, verify ALL new procedures are registered. Default-deny means unregistered = broken.
2. Check auth middleware — verify tenant isolation / org scoping is still wired correctly.
3. For every new backend route or procedure, verify:
- All mutations require authentication
- Input validation uses schema validation (Zod, Joi, etc.), not raw input
- No raw SQL without parameterization (string concatenation in queries)
4. For every new API route or webhook handler, verify:
- Webhook routes validate signatures (HMAC, etc.)
- Auth checks present
- No secrets in code
5. Check for:
- process.env usage in frontend/UI packages (should use typed env config)
- Error messages that leak internals in production (must guard with NODE_ENV)
- Missing CSRF/CORS protections on mutations
- timingSafeEqual for any signature comparisons (not === )
- Secrets or tokens in URL parameters
## OUTPUT FORMAT (MANDATORY)
SECURITY AUDIT REPORT
=====================
Scope: [service/iterations audited]
Findings: [count by severity]
HIGH:
- [H-NNN]: [title] | [file:line] | [description]
MEDIUM:
- [M-NNN]: [title] | [file:line] | [description]
LOW:
- [L-NNN]: [title] | [file:line] | [description]
CROSS-REF:
- [Agent N]: [what they should verify]
Auth Coverage: [summary of auth/RBAC status]
Webhook Security: [all validated / gaps / N/A]
Agent 2: Architecture Auditor
{context block}
{agent roster}
## Your Focus: Architecture & Pattern Conformance
Verify Ralph followed established patterns:
1. Read AGENTS.md (or CLAUDE.md) — these define the canonical patterns
2. For every new file Ralph created, verify:
- Files are in the correct directories per project conventions
- Import conventions are followed (package aliases, no cross-boundary relative imports)
- Established patterns are used (not reinvented)
3. Check dependency graph:
- No circular dependencies between packages/modules
- Dependency direction matches architecture (leaf packages don't import from app layer)
4. Naming conventions followed (routes, files, variables)
5. New modules/routers/services registered in their respective index/root files
## OUTPUT FORMAT (MANDATORY)
ARCHITECTURE AUDIT REPORT
=========================
Scope: [service/iterations audited]
Findings: [count by severity]
HIGH:
- [H-NNN]: [title] | [file] | [description]
MEDIUM:
- [M-NNN]: [title] | [file] | [description]
LOW:
- [L-NNN]: [title] | [file] | [description]
CROSS-REF:
- [Agent N]: [what they should verify]
Pattern Conformance: [PASS/PARTIAL/FAIL]
Dependency Graph: [CLEAN/VIOLATIONS]
Registration: [all registered / total new modules]
Agent 3: Data Layer Auditor
{context block}
{agent roster}
## Your Focus: Database & Data Integrity
Audit schema changes, migrations, and data access patterns:
1. Read the schema file(s) — check for:
- Proper column types (numeric precision, enum usage)
- FK relationships defined
- Indexes on frequently queried columns
- Tenant isolation columns on all tenant-scoped tables (if multi-tenant)
2. Read any new migration files:
- Wrapped in transactions (BEGIN/COMMIT)?
- Idempotent (IF NOT EXISTS, IF EXISTS)?
- Data migration for column type changes?
3. Check query patterns in backend routes/services:
- Tenant-scoped queries include tenant filter
- Pagination present on list queries (cursor-based preferred over offset)
- No N+1 patterns (batch operations, not loops)
- Proper error handling on DB operations
4. Type helpers / inference types exported where applicable
## OUTPUT FORMAT (MANDATORY)
DATA LAYER AUDIT REPORT
=======================
Scope: [service/iterations audited]
Findings: [count by severity]
HIGH:
- [H-NNN]: [title] | [file:line] | [description]
MEDIUM:
- [M-NNN]: [title] | [file:line] | [description]
LOW:
- [L-NNN]: [title] | [file:line] | [description]
CROSS-REF:
- [Agent N]: [what they should verify]
Schema Integrity: [PASS/PARTIAL/FAIL]
Migration Quality: [PASS/PARTIAL/FAIL]
Tenant Isolation: [PASS/PARTIAL/FAIL/N/A]
Agent 4: UI/UX Auditor
{context block}
{agent roster}
## Your Focus: UI/UX Quality
**Skip this audit entirely if the project has no frontend/UI layer. Report "N/A — no frontend" and exit.**
Audit frontend code for consistency and completeness:
1. For every new page:
- Does it have a loading state (skeleton, spinner, loading.tsx)?
- Does it have proper empty state handling?
- Does it have error boundary coverage?
2. For every new component:
- Uses the project's UI component library (not raw HTML for things the library covers)
- Dark mode compatible (CSS variables, not hardcoded colors) — if applicable
- Accessible (aria-labels on interactive elements)
- Mobile-responsive (no fixed widths)
3. Check navigation — are new pages linked in nav/sidebar?
4. Forms use the project's form pattern (e.g., react-hook-form + Zod, Formik, etc.)
5. User feedback via toast/notification utility (not window.alert)
6. Error messages use dev-only guards for stack traces
## OUTPUT FORMAT (MANDATORY)
UI/UX AUDIT REPORT
==================
Scope: [service/iterations audited]
Findings: [count by severity]
HIGH:
- [H-NNN]: [title] | [file] | [description]
MEDIUM:
- [M-NNN]: [title] | [file] | [description]
LOW:
- [L-NNN]: [title] | [file] | [description]
CROSS-REF:
- [Agent N]: [what they should verify]
Pages Audited: [count]
Missing loading states: [list or "none"]
Missing empty states: [list or "none"]
Accessibility: [PASS/PARTIAL/FAIL]
Agent 5: Integration Auditor
{context block}
{agent roster}
## Your Focus: Cross-Package Integration
Verify everything wires together correctly:
1. For every new backend route/procedure:
- Registered in the root router/index
- Registered in access control (RBAC matrix, permissions, etc.) if applicable
- Input schemas importable from shared types package
2. For every new page that calls the backend:
- Correct API paths / procedure names
- Error handling for API errors
- Loading states while queries resolve
3. Cross-package type safety:
- DB types flow to backend output types
- Backend output types flow to frontend components
- No `any` types at package boundaries
4. Environment variables:
- New vars added to both typed env config AND .env.example
- Optional vs required matches actual usage
5. API contract compliance (if api-contract.md or similar exists):
- Response shapes match the contract spec
- Status codes match documented behavior
- Error formats are consistent
6. Build pipeline:
- New packages detected by build tool (turbo, nx, etc.)
- No imports that would break at build time
## OUTPUT FORMAT (MANDATORY)
INTEGRATION AUDIT REPORT
========================
Scope: [service/iterations audited]
Findings: [count by severity]
HIGH:
- [H-NNN]: [title] | [file] | [description]
MEDIUM:
- [M-NNN]: [title] | [file] | [description]
LOW:
- [L-NNN]: [title] | [file] | [description]