tharunramagiri/muster-teams/experiment-review

Experiment Review

Use this when

A marketing experiment has run long enough to produce data and needs a structured review.

Process

1. State the original hypothesis, expected outcome, and decision criteria before reviewing data.
2. Present the actual results with the primary metric, supporting diagnostics, sample size, and time window.
3. Analyze whether the result is statistically meaningful or if more data is needed.
4. Separate correlation from causation — did the experiment actually cause the observed change?
5. Make a clear decision: scale, iterate with changes, or kill — with reasoning.
6. Document learnings and recommend the next experiment based on evidence.

Output

An experiment review with: hypothesis, results, analysis, decision (scale/iterate/kill), reasoning, learnings, and next recommended experiment.

Guardrails

Never defend a losing experiment past its decision threshold. Be honest about what the data does and does not show. Avoid false precision — round appropriately and state confidence levels.