ai4scientist/nano-scientist/rebuttal

Workflow 4: Rebuttal

Prepare and maintain a grounded, venue-compliant rebuttal for: $ARGUMENTS

Scope

This skill is optimized for:

• text-only rebuttal under strict character/word limits (e.g. ICML single-document)
• per-reviewer thread responses where each reviewer renders independently (e.g. OpenReview-style)
• multiple reviewers with shared and reviewer-specific concerns
• follow-up rounds after the initial rebuttal
• safe drafting with no fabrication, no overpromise, and full issue coverage

This skill does not:

• run new experiments automatically
• generate new theorem claims automatically
• edit or upload a revised PDF
• submit to OpenReview / CMT / HotCRP

If the user already has new results, derivations, or approved commitments, the skill can incorporate them as user-confirmed evidence.

Lifecycle Position

Workflow 1:   idea-discovery
Workflow 1.5: experiment-bridge
Workflow 2:   auto-review-loop (pre-submission)
Workflow 3:   paper-writing
Workflow 4:   rebuttal (post-submission external reviews)

Constants

• VENUE = `ICML` — Default venue. Override if needed.
• RESPONSE_MODE = `TEXT_ONLY` — v1 default.
• REVIEWER_MODEL = `gpt-5.4` — Used via Codex MCP for internal stress-testing.
• REVIEWER_BACKEND = `codex` — Default: Codex MCP (xhigh). Override with — reviewer: oracle-pro for GPT-5.4 Pro via Oracle MCP. See shared-references/reviewer-routing.md.
• MAX_INTERNAL_DRAFT_ROUNDS = 2 — draft → lint → revise.
• VENUE_MODE = `single_document` — single_document for one shared author response, or per_reviewer_thread when each reviewer thread renders independently. Confirm the venue/interface before drafting if unclear. Affects Phase 4/7 output shape.
• STRESS_TEST_ROUNDS_BASE = 1 — One Codex MCP critique round on the full response set. Add focused rounds for reviewer_priority: pivotal responses, terminating when Codex returns no new substantive issues. Hard cap at 5.
• MAX_FOLLOWUP_ROUNDS = 3 — per reviewer thread.
• AUTO_EXPERIMENT = false — When true, automatically invoke /experiment-bridge to run supplementary experiments when the strategy plan identifies reviewer concerns that require new empirical evidence. When false (default), pause and present the evidence gap to the user for manual handling.
• QUICK_MODE = false — When true, only run Phase 0-3 (parse reviews, atomize concerns, build strategy). Outputs ISSUE_BOARD.md + STRATEGY_PLAN.md and stops — no drafting, no stress test. Useful for quickly understanding what reviewers want before deciding how to respond.
• REBUTTAL_DIR = `rebuttal/`

Override: /rebuttal "paper/" — venue: NeurIPS, character limit: 5000

Required Inputs

1. Paper source — PDF, LaTeX directory, or narrative summary
2. Raw reviews — pasted text, markdown, or PDF with reviewer IDs
3. Venue rules — venue name, character/word limit, text-only or revised PDF allowed, rendering mode (one shared response or independent reviewer threads)
4. Current stage — initial rebuttal or follow-up round

If venue rules, limit, or rendering mode are missing, stop and ask before drafting.

Safety Model

Three hard gates — if any fails, do NOT finalize:

1. Provenance gate — every factual statement maps to: paper, review, user_confirmed_result, user_confirmed_derivation, or future_work. No source = blocked.
2. Commitment gate — every promise maps to: already_done, approved_for_rebuttal, or future_work_only. Not approved = blocked.
3. Coverage gate — every reviewer concern ends in: answered, deferred_intentionally, or needs_user_input. No issue disappears.

Workflow

Phase 0: Resume or Initialize

1. If rebuttal/REBUTTAL_STATE.md exists → resume from recorded phase
2. Otherwise → create rebuttal/, initialize all output documents
3. Load paper, reviews, venue rules, any user-confirmed evidence

Phase 1: Validate Inputs and Normalize Reviews

1. Validate venue rules are explicit
2. Normalize all reviewer text into rebuttal/REVIEWS_RAW.md (verbatim)
3. Record metadata in rebuttal/REBUTTAL_STATE.md
4. If ambiguous, pause and ask

Phase 2: Atomize and Classify Reviewer Concerns

Create rebuttal/ISSUE_BOARD.md.

For each atomic concern:

• issue_id (e.g., R1-C2)
• reviewer, round, raw_anchor (short quote)
• issue_type: assumptions / theorem_rigor / novelty / empirical_support / baseline_comparison / complexity / practical_significance / clarity / reproducibility / other
• severity: critical / major / minor
• reviewer_stance: positive / swing / negative / unknown
• reviewer_priority: standard / pivotal
• pivotal — a reviewer whose response is likely to affect the decision if addressed well: low or borderline rating, addressable concerns, and enough confidence/influence to matter. Phase 3 allocates extra drafting and stress-test budget here.
• response_mode: direct_clarification / grounded_evidence / nearest_work_delta / assumption_hierarchy / narrow_concession / future_work_boundary / structural_distinction
• structural_distinction — for "your method reduces to X / is just generic Y / is subsumed by Z" attacks. Pattern: agree on the local reduction; show the structural feature your parameterization preserves that X/Y/Z does not capture, backed by a concrete mechanism (theorem dependency, derivation step, or empirical consequence). Never use rhetorically without the supporting mechanism.
• status: open / answered / deferred / needs_user_input

Phase 3: Build Strategy Plan

Create rebuttal/STRATEGY_PLAN.md.

1. Identify 2-4 global themes resolving shared concerns
2. Choose response mode per issue
3. Build character budget (10-15% opener, 75-80% per-reviewer, 5-10% closing) — applies in single_document mode; in per_reviewer_thread mode, set per-thread word/char targets instead
4. Identify pivotal reviewer(s) — reviewers whose vote or confidence shift would most affect the decision, especially when concerns are addressable rather than ideological. Mark them reviewer_priority: pivotal in ISSUE_BOARD.md. There may be more than one. Allocate disproportionate drafting + stress-test budget here.
5. Identify blocked claims (ungrounded or unapproved)
6. If unresolved blockers → pause and present to user

QUICK_MODE exit: If QUICK_MODE = true, stop here. Present ISSUE_BOARD.md + STRATEGY_PLAN.md to the user and summarize: how many issues per reviewer, shared vs unique concerns, recommended priorities, and evidence gaps. The user can then decide to continue with full rebuttal (/rebuttal — quick mode: false) or write manually.

Phase 3.5: Evidence Sprint (when AUTO_EXPERIMENT = true)

Skip entirely if `AUTO_EXPERIMENT` is `false` — instead, pause and present the evidence gaps to the user.

If the strategy plan identifies issues that require new empirical evidence (tagged response_mode: grounded_evidence with evidence_source: needs_experiment):

1. Generate a mini experiment plan from the reviewer concerns:

• What to run (ablation, baseline comparison, scale-up, condition check)
• Success criterion (what result would satisfy the reviewer)
• Estimated GPU-hours

1. Invoke /experiment-bridge with the mini plan:

`` /experiment-bridge "rebuttal/REBUTTAL_EXPERIMENT_PLAN.md" ``

1. Wait for results, then update ISSUE_BOARD.md:

• Tag completed experiments as user_confirmed_result
• Update evidence source for relevant issue cards

1. If experiments fail or are inconclusive:

• Switch response mode to narrow_concession or future_work_boundary
• Do NOT fabricate positive results

1. Save experiment results to rebuttal/REBUTTAL_EXPERIMENTS.md for provenance tracking.

Time guard: If estimated GPU-hours exceed rebuttal deadline, skip and flag for manual handling.

Phase 4: Draft Initial Rebuttal

Create the draft artifact(s) per VENUE_MODE:

• single_document mode → one rebuttal/REBUTTAL_DRAFT_v1.md
• per_reviewer_thread mode → one rebuttal/Reviewer_<ID>_response.md per reviewer (no top-level REBUTTAL_DRAFT_v1.md)

Structure depends on `VENUE_MODE`:

• single_document — one REBUTTAL_DRAFT_v1.md:

1. Short opener — thank reviewers + 2-4 global resolutions
2. Per-reviewer numbered responses — answer → evidence → implication
3. Short closing — resolved / remaining / acceptance case

• per_reviewer_thread — one self-contained Reviewer_<ID>_response.md per reviewer:

1. Brief acknowledgment of that reviewer's main thrust
2. Numbered W#/Q# responses (answer → evidence → implication)
3. Optional shared experimental-setup paragraph (see "Reusable setup block" below)

• Each file must be readable standalone. No "see Reviewer X's response" references. No global opener.

Default reply pattern per issue:

• Sentence 1: direct answer
• Sentence 2-4: grounded evidence
• Last sentence: implication for the paper

Reusable setup block (per_reviewer_thread mode). If multiple reviewer-thread responses need the same experimental setup or metric definitions, write a canonical SETUP_METRICS_BLOCK.md. Reuse it consistently in each reviewer file that needs it. Target ≤ 150 words; expand only with genuinely reviewer-specific additions inline. Change-once-update-everywhere prevents drift across threads.

Heuristics from successful rebuttals (content):

• Evidence > assertion
• Global narrative first, per-reviewer detail second (single_document mode only)
• Concrete numbers for counter-intuitive points
• Name closest prior work + exact delta for novelty disputes
• Concede narrowly when reviewer is right
• For theory: separate core vs technical assumptions
• Answer friendly reviewers too

Reviewer-defensive moves:

• Minimum sufficient evidence per concern. Usually one numerical anchor: the metric that maps directly to that reviewer's specific ask. Cut metrics other reviewers care about — bloat dilutes the answer.
• Pre-registered calibration phrasing. When a threshold or hold-out was fixed before generated samples were inspected, say so explicitly with a phrase like "set on hold-out before any generated sample was inspected." Defuses cherry-pick attacks at near-zero word cost. Only use when actually true.
• Surface non-obvious design choices upfront. If the experimental setup has a non-obvious caveat (compute-matched ≠ epoch-matched, atypical seed protocol, restricted parameter subset, etc.), name it concretely with numbers where they clarify the design choice. Pre-empts adversarial reverse engineering.
• Structural distinction over denial. When a reviewer claims your work reduces to / is subsumed by a generic framework, do not deny the reduction. Identify the structural feature your parameterization preserves that the generic framework does not — see response_mode: structural_distinction.
• Concede without surrendering the claim. When the reviewer is partly right, explicitly accept the local point, then state what remains true and why it still supports the paper's contribution. Pair the concession with the preserved theorem, mechanism, empirical result, or scope condition.

Hard rules:

• NEVER invent experiments, numbers, derivations, citations, or links
• NEVER promise what user hasn't approved
• If no strong evidence exists, say less not more

`single_document` mode only: also generate rebuttal/PASTE_READY.txt (plain text, exact character count for the OpenReview/CMT paste field). In per_reviewer_thread mode skip this artifact — each Reviewer_<ID>_response.md is itself the paste target for its thread.

Also generate rebuttal/REVISION_PLAN.md — the overall revision checklist.