timelovercc/mojito/research-review

Research Review

This is a generic example skill shipped with Mojito; adapt the checks to your field.

Pick the review by the artifact and the decision at stake, read the artifact together with its evidence, and keep the research goal in view.

Early exploration sketches get fact checks and useful questions, not a full novelty gate. Engineering smoke tests and intermediate diagnostics need no independent review. Re-review when a load-bearing claim or its evidence changes, not for formatting.

Idea review

• Restate the question and the capability it serves. Is it the question the idea card actually asks?
• Find the nearest prior work yourself and compare problem, setting, method and evidence. Name exact overlaps.
• List rival explanations the author did not rule out, and the cheapest observation that would separate them.
• Judge importance, novelty, measurement trust and feasibility separately; do not merge them into one score.

Claim-evidence audit

For every claim that carries weight:

1. Find the evidence it rests on: the run, table, figure or citation. Open the original, not a summary of it.
2. Check that the number, the setting and the comparison match what the claim says.
3. Check the measurement: metric bugs, judge behavior, comparable baselines and budgets, answer distributions, held-out results.
4. Check the scope: does a small or synthetic result get stated as general?
5. Label each claim supported, unsupported, overstated (narrow it), contradicted, or missing evidence (say which run would settle it).

How to review

Reviewers are independent of the author: give them the actual artifact, its sources, the tools they may use and where they may write. If a required reviewer is not available, report the missing review instead of claiming a substitute ran.

The lead checks every consequential finding against the original source. Tell apart a factual error, an unsupported claim, an alternative explanation, an exact prior-work overlap and missing evidence. For each, say which judgment changes and the smallest repair or diagnostic. Keep honest negative results. A high rejection rate is not a measure of review quality.

Write the review into the existing record (the idea's evidence in Idea Lab, or the paper's review notes), linked to the reviewed version. A review can refute a claim or recommend a change of direction; it does not by itself stop a project. After revision, recheck the affected claims and note what changed.