empiricaai/empirica/eat-the-broccoli

Eat the Broccoli 🥦

The thorough quality sweep you run before a release, after a refactor, or when something just smells off. Two halves:

1. Deterministic tooling — lint, types, tests, dependency CVEs, dead code,

silent failures. Catches the known categories.

1. The pattern hunt — a checklist of the hard failure classes that pass

every test and still ship broken: races, stale caches, silent fallbacks, null-masking, deploy-staleness. Tools can't catch these; only a trained eye plus a "is this by design?" check can.

Portable by design. The deterministic half is tool-agnostic — wire in your

stack's linter / typechecker / scanner. The pattern hunt needs no tooling at

all; it's a structured way of looking.

Two dials: depth × scope

Pick any combination — quick × changed is a pre-commit reflex; deep × repo is a pre-release gate. State the level + scope you chose, and say what you skipped — silent truncation reads as "covered everything."

Forks & monorepos: when most of the tree is upstream/vendored, repo

scope drowns in code you don't own. Default to changed or your owned

packages — here the scope dial is load-bearing, not optional.

Phase 1 — Deterministic tooling

Run your stack's equivalent of each. Skip what you don't have; the sweep still works.

On that last row — coverage does not cover it. Coverage asks "was this line executed?"; test freshness asks "could this test have failed?" Those come apart badly in fast iterative work, because tests written against yesterday's contract keep passing against today's code and actively guard the drift: a green assertion that the API returns 422 is indistinguishable from a green assertion that it should. Fixtures are the sharpest edge — a hand-built schema that has fallen behind the real one makes fixture and code agree with each other and disagree with production, so the suite is green precisely where it is blind. If a defect survives a green suite, the suite is a suspect, not an alibi.

Tooling by language

Concrete per-stack tool choices (Python, Rust) live in references/stacks.md — consult it when wiring a stack, not while hunting. The pattern hunt below is language-agnostic and ports verbatim across any stack.

Reading silent failures: a truly-silent swallow (except: pass, empty catch {}, let _ = fallible()) is the dangerous one. A broad catch that logs or re-raises is often deliberate degradation — flag the new ones, and any with no log and no re-raise, not the absolute count.

Phase 2 — The pattern hunt 🥦 (the part that earns the name)

Tools find known categories. These are the structural failures that pass tests and still ship broken. Each is a place where broken and by-design look identical — so each row carries the disambiguator that tells them apart.

The one meta-question: "This looks intentional — is it actually?" When you find a smell, check the intent (a comment, a test, a doc, or ask). When it resolves to "yes, by design," record that verdict (an inline annotation or a .broccoli-accept line — see below) so you don't re-litigate it next sweep.

Worked example. You spot a function returning [] on a DB timeout (fallback-masks-primary). Check the intent: does it log or raise? It logs nothing → ❌ broken — callers can't tell "no rows" from "DB down." If instead it logged a warning and returned [] as a documented degraded mode → ✅ by design: record it with the reason, move on.

A. State & timing

B. Failure visibility