naver/fixture-monkey/write-fixture

Writing tests with Fixture Monkey

Fixture Monkey generates test objects with random values. Two skills matter: choosing which cases to test, and knowing what not to set in each one.

Pin the properties that force the expected outcome. Leave everything else random.

A fixture that sets every field is a hand-rolled builder with extra ceremony — it breaks when an unrelated field is added, and it drags the test file into diffs it has nothing to do with.

Scope: tests only

Never modify production code. This work adds and edits tests. Production sources are read-only — read them to understand behaviour, never to change it.

That holds even when changing them would be easier:

If a test genuinely cannot be written without a production change, stop and say so, naming the change and why it is needed. Let the user decide. A production edit smuggled in with a test is the one thing a reviewer will not be looking for.

Procedure

Steps 1–4 decide which tests to write. Steps 5–6 write each one.

1. Identify the method under test

Name the exact method. Read its body and the types it takes and returns. If the request is vague ("write tests for the order service"), narrow it to specific methods and say which ones you picked.

Everything downstream depends on this being one concrete method, not a class or a feature.

2. Enumerate the cases the method can produce

Work through the body and list every distinct outcome. Look for:

• Branches — every if / else / when / switch arm, and every guard clause.
• Boundaries — for each comparison, the value below, at, and above the threshold. amount >= 100_000 yields 99,999 / 100,000 / 100,001.
• Exceptions — every throw, and every call that can throw.
• Empty and absent — empty collection, null, Optional.empty(), zero, when the type permits them.
• Enum and subtype fan-out — each constant or implementation that reaches a different path.
• Interactions — combinations where two conditions are not independent, e.g. a member discount and a coupon that cannot stack.

List them plainly before writing code. This list is the deliverable of steps 2–3, and it is worth showing to the user.

3. Select the cases worth testing

Do not test everything you listed. Keep a case if it earns its place:

Aim for the smallest set that covers every outcome once, plus the boundaries. Say which cases you dropped and why — that reasoning is what lets the user disagree.

4. Ask before going exhaustive

The step-3 list is the default: representative coverage. Exhaustive coverage — every combination, every boundary, every enum constant — costs real test-suite time and maintenance, so it is the user's call, not yours.

If the user has already asked for thorough or exhaustive tests, skip the question and cover the full step-2 list. Otherwise, present the selected cases, note roughly how many more a full sweep would add, and ask whether they want it. Then proceed with the answer — do not block on it if the user is not present; write the representative set and note that the exhaustive set is available.

When exhaustive coverage is wanted, prefer @ParameterizedTest over copy-pasted methods, so the case list stays readable.

5. Identify the properties that force the outcome

For each selected case, pin a property only if one of these holds:

Everything else stays random. Do not pin because a property is conceptually important, because production requires it, or because a random value looks odd in a debugger — none of that changes the outcome.

The test for a pin: if this property were random, could the assertion still be written as an exact expected value? If yes, leave it random.

Over-pinning is the common failure and no test run catches it — it surfaces later as a diff in an unrelated pull request. When unsure, leave it random; step 6's verification will tell you if you were wrong.

When two properties must agree with each other, do not pin both to hand-computed constants — pin the minimum set and derive the rest with thenApply.

6. Build the fixture with set, then verify

Pin with the narrowest API that expresses the constraint. Take the first row that fits:

A value that merely has to be legal is not a pin — it is a domain constraint, and it belongs somewhere the next test does not have to repeat it. See Constrain the values you did not pin below.

Two ordering rules, both silent when broken.

size before element writes, or the elements land on a collection that may be too short and are dropped.

And on overlapping paths the last call wins — including across a parent and its child, where it decides whether the parent exists at all:

.setNull(javaGetter(Order::getCustomer))                                  // parent nulled
.set(javaGetter(Order::getCustomer).into(Customer::getName), "Kim")       // parent revived, name set
 
.set(javaGetter(Order::getCustomer).into(Customer::getName), "Kim")
.setNull(javaGetter(Order::getCustomer))                                  // parent null; the name pin is gone

The first order is what makes a shared base fixture work — null a subtree there, revive just the part a case needs. The second loses the pin with no exception. A helper that returns a builder gives the right order for free, since the case's own calls come after.

`Values.just` is for when the value must not be rebuilt — nothing else. Plain set decomposes the object and regenerates its properties, so a pre-built list comes back with different element values and no exception is raised. That silence is a symptom to recognise, not a reason to reach for Values.just by default: if the assertion does not depend on those inner values, plain set is correct and leaves the rest random, which is the point. Reach for Values.just only when the exact instance is what the case is about — the same restraint that applies to fixed().

Select properties type-safely. String paths break silently on rename, and an unmatched path is ignored rather than reported.

.set(javaGetter(Order::getStatus), OrderStatus.PAID)     // Java

.setExp(Order::status, OrderStatus.PAID)                 // Kotlin

javaGetter resolves the property name by stripping a get/is prefix, or by using the method name as-is when it already matches a field. Record accessors and other prefix-less getters work unchanged — javaGetter(Metric::totalChangeCount) is as valid as javaGetter(Order::getStatus), including for primitive components. A fluent accessor whose name does not match any field is the one case that fails to resolve.

Java imports javaGetter from com.navercorp.fixturemonkey.api.expression.JavaGetterMethodPropertySelector — the identically named one in ...api.experimental is deprecated. Kotlin needs KotlinPlugin and takes the Exp suffix on every builder method (setExp, sizeExp, setNullExp, setNotNullExp, setPostConditionExp), with an ExpGetter variant for Java-style getter references.

Nullability comes first

Every nullable property is null 20% of the time by default (DEFAULT_NULL_INJECT = 0.2). That single fact causes more confusing failures than anything else here: production code does projects.stream().collect(toMap(Project::getId, ...)), getId() comes back null, and the NPE looks like a service bug.

So before pinning anything else: any field the code under test dereferences must be non-null. Decide it explicitly rather than hoping.

`defaultNotNull(true)` only decides what the type left unsaid. Declared nullability outranks it, and nothing warns you when it loses:

Highest wins: @NotNull / @NonNull → @Nullable or Kotlin ? → defaultNotNull(true) → the 0.2 default. So the annotation that documents "this may be absent" is precisely what keeps the flakiness, and @Nullable here means any name in the default list — javax.annotation, jakarta.annotation, org.springframework.lang, org.checkerframework.checker.nullness.qual, org.jspecify.annotations, org.eclipse.jgit.annotations, org.jmlspecs.annotation.

The annotation is production code, so it stays. Fix it on the fixture side: