version: "1.0.0" name: adversarial-test-design description: >- Non-obvious test-design failures where tests pass but wouldn't catch a real bug: implementation-mirroring tests, assertions too weak to fail, and missing boundary/negative cases. Use when writing or reviewing tests for correctness, right after implementation code, or when a test suite is green but confidence in it is unclear. Does not cover the red-green-refactor process itself — see test-driven-development for that.

Adversarial Test Design

Green tests aren't proof of correctness. Each entry: how a test passes despite the code being wrong.

Implementation-mirroring tests

Wrong: The test re-derives the same computation the code does and compares the two, or asserts on internal steps rather than the observable outcome. E.g. testing a sort function by checking that compare(a,b) was called, not that the output is sorted.

Right: Assert against an independently-known expected value or an invariant of the output, not against a recomputation of the implementation's logic.

Why: If the test derives its expected value the same way the code does, a bug in the shared reasoning shows up in both and the test still passes — the test is checking "does the code agree with itself," not "is the code right." This is the single most common way a green suite ships a real bug: it doesn't lie, it just isn't asking the right question.

Assertions too weak to fail

Wrong: expect(result).toBeDefined(), expect(response.status).not.toBe(500), expect(Array.isArray(result)).toBe(true).

Right: Assert the actual value/shape/count that the operation is supposed to produce: expect(result).toEqual([...]), expect(response.status).toBe(200), expect(result.length).toBe(3).

Why: A test that checks "is truthy" or "is not the one specific error I thought of" passes for almost any output, including wrong ones. Self-check before trusting a green test: pick one line of the implementation, invert its logic (flip a condition, swap an operator, off-by-one an index), and ask whether this specific test would fail. If the answer is no, the assertion isn't doing anything.

Boundary and negative cases skipped by default

Wrong: Tests cover only the intended-use, happy-path input (a valid, medium-sized, well-formed example).

Right: For any input, also cover: empty (empty string/array/zero items), exactly one, the off-by-one boundary of any range/limit check, malformed/wrong-type input, and — for anything touched by more than one caller — two concurrent callers.

Why: These are precisely the cases where a plausible-looking implementation diverges from a correct one (< vs <=, length vs length - 1, unhandled empty collection short-circuiting differently than expected). They're also exactly the cases a model tends to skip when generating tests for its own code, because the code it just wrote was shaped around the happy path it was thinking about.

The self-check before trusting a suite

Before considering a test suite sufficient, ask of each test: if I subtly broke this specific piece of logic, would this test go red? If you can't name which test would catch a given mutation (an inverted condition, a swapped operand, a dropped edge case), that logic isn't actually covered — regardless of code-coverage percentage.