jiangxinke/harness-rl/academic-paper-review
Claude Code and agent skill by jiangxinke/harness-rl
- community
Checking that the SKILL.md file is still in the repository.
Use this skill when the user requests to review, analyze, critique, or summarize academic papers, research articles, preprints, or scientific publications. Supports comprehensive structured reviews covering methodology assessment, contribution evaluation, literature positioning, and constructive feedback generation. Trigger on queries involving paper URLs, uploaded PDFs, arXiv links, or requests like "review this paper", "analyze this research", "summarize this study", or "write a peer review".
Install
npx vskill@latest install jiangxinke/harness-rl/academic-paper-reviewSKILL.md
The same SKILL.md was published earlier as jiangxinke/harness-rl/academic-paper-review.
Academic Paper Review Skill
Overview
This skill produces structured, peer-review-quality analyses of academic papers and research publications. It follows established academic review standards used by top-tier venues (NeurIPS, ICML, ACL, Nature, IEEE) to provide rigorous, constructive, and balanced assessments.
The review covers summary, strengths, weaknesses, methodology assessment, contribution evaluation, literature positioning, and actionable recommendations — all grounded in evidence from the paper itself.
Core Capabilities
- Parse and comprehend academic papers from uploaded PDFs or fetched URLs
- Generate structured reviews following top-venue review templates
- Assess methodology rigor (experimental design, statistical validity, reproducibility)
- Evaluate novelty and significance of contributions
- Position the work within the broader research landscape via targeted literature search
- Identify limitations, gaps, and potential improvements
- Produce both detailed review and concise executive summary formats
- Support papers in any scientific domain (CS, biology, physics, social sciences, etc.)
When to Use This Skill
Always load this skill when:
- User provides a paper URL (arXiv, DOI, conference proceedings, journal link)
- User uploads a PDF of a research paper or preprint
- User asks to "review", "analyze", "critique", "assess", or "summarize" a research paper
- User wants to understand the strengths and weaknesses of a study
- User requests a peer-review-style evaluation of academic work
- User asks for help preparing a review for a conference or journal submission
Review Methodology
Phase 1: Paper Comprehension
Thoroughly read and understand the paper before forming any judgments.
Step 1.1: Identify Paper Metadata
Extract and record:
| Field | Description | |
|---|---|---|
| Title | Full paper title | |
| Authors | Author list and affiliations | |
| Venue / Status | Publication venue, preprint server, or submission status | |
| Year | Publication or submission year | |
| Domain | Research field and subfield | |
| Paper Type | Empirical, theoretical, survey, position paper, systems paper, etc. |
Step 1.2: Deep Reading Pass
Read the paper systematically:
- Abstract & Introduction — Identify the claimed contributions and motivation
- Related Work — Note how authors position their work relative to prior art
- Methodology — Understand the proposed approach, model, or framework in detail
- Experiments / Results — Examine datasets, baselines, metrics, and reported outcomes
- Discussion & Limitations — Note any self-identified limitations
- Conclusion — Compare concluded claims against actual evidence presented
Step 1.3: Key Claims Extraction
List the paper's main claims explicitly:
Phase 2: Critical Analysis
Step 2.1: Literature Context Search
Use web search to understand the research landscape:
Use web_fetch on key related papers or surveys to understand where this work fits.
Step 2.2: Methodology Assessment
Evaluate the methodology using the following framework:
| Criterion | Questions to Ask | Rating | |
|---|---|---|---|
| Soundness | Is the approach technically correct? Are there logical flaws? | 1-5 | |
| Novelty | What is genuinely new vs. incremental improvement? | 1-5 | |
| Reproducibility | Are details sufficient to reproduce? Code/data available? | 1-5 | |
| Experimental Design | Are baselines fair? Are ablations adequate? Are datasets appropriate? | 1-5 | |
| Statistical Rigor | Are results statistically significant? Error bars reported? Multiple runs? | 1-5 | |
| Scalability | Does the approach scale? Are computational costs discussed? | 1-5 |
Step 2.3: Contribution Significance Assessment
Evaluate the significance level:
| Level | Description | Criteria | |
|---|---|---|---|
| Landmark | Fundamentally changes the field | New paradigm, widely applicable breakthrough | |
| Significant | Strong contribution advancing the state of the art | Clear improvement with solid evidence | |
| Moderate | Useful contribution with some limitations | Incremental but valid improvement | |
| Marginal | Minimal advance over existing work | Small gains, narrow applicability | |
| Below threshold | Does not meet publication standards | Fundamental flaws, insufficient evidence |
Step 2.4: Strengths and Weaknesses Analysis
For each strength or weakness, provide:
- What: Specific observation
- Where: Section/figure/table reference
- Why it matters: Impact on the paper's claims or utility
Phase 3: Review Synthesis
Step 3.1: Assemble the Structured Review
Produce the final review using the template below.
Review Output Template
Review Principles
Constructive Criticism
- Always suggest how to fix it — Don't just point out problems; propose solutions
- Give credit where due — Acknowledge genuine contributions even in flawed papers
- Be specific — Reference exact sections, equations, figures, and tables
- Separate minor from major — Distinguish fatal flaws from fixable issues
Objectivity Standards
- ❌ "This paper is poorly written" (vague, unhelpful)
- ✅ "Section 3.2 introduces notation X without formal definition, making the proof in Theorem 1 difficult to follow. Consider adding a notation table after the problem formulation." (specific, actionable)
Ethical Review Practices
- Do NOT dismiss work based on author reputation or affiliation
- Evaluate the work on its own merits
- Flag potential ethical concerns (bias in datasets, dual-use implications) constructively
- Maintain confidentiality of unpublished work
Adaptation by Paper Type
| Paper Type | Focus Areas | |
|---|---|---|
| Empirical | Experimental design, baselines, statistical significance, ablations, reproducibility | |
| Theoretical | Proof correctness, assumption reasonableness, tightness of bounds, connection to practice | |
| Survey | Comprehensiveness, taxonomy quality, coverage of recent work, synthesis insights | |
| Systems | Architecture decisions, scalability evidence, real-world deployment, engineering contributions | |
| Position | Argument coherence, evidence for claims, impact potential, fairness of characterizations |
Common Pitfalls to Avoid
- ❌ Reviewing the paper you wish was written instead of the paper that was submitted
- ❌ Demanding additional experiments that are unreasonable in scope
- ❌ Penalizing the paper for not solving a different problem
- ❌ Being overly influenced by writing quality versus technical contribution
- ❌ Treating absence of comparison to your own work as a weakness
- ❌ Providing only a summary without critical analysis
Quality Checklist
Before finalizing the review, verify:
- [ ] Paper was read completely (not just abstract and introduction)
- [ ] All major claims are identified and evaluated against evidence
- [ ] At least 3 strengths and 3 weaknesses are provided with specific references
- [ ] The methodology assessment table is complete with ratings and justifications
- [ ] Questions for authors target genuine ambiguities, not rhetorical critiques
- [ ] Literature search was conducted to contextualize the contribution
- [ ] Recommendations are actionable and constructive
- [ ] The overall assessment is consistent with the identified strengths and weaknesses
- [ ] The review tone is professional and respectful
- [ ] Minor issues are separated from major concerns
Output Format
- Output the complete review in Markdown format
- Save the review to
/mnt/user-data/outputs/review-{paper-topic}.mdwhen working in sandbox - Present the review to the user using the
present_filestool
Notes
- This skill complements the
deep-researchskill — load both when the user wants the paper reviewed in the context of the broader field - For papers behind paywalls, work with whatever content is accessible (abstract, publicly available versions, preprint mirrors)
- Adapt the review depth to the user's needs: a brief assessment for quick triage versus a full review for submission preparation
- When reviewing multiple papers comparatively, maintain consistent criteria across all reviews
- Always disclose limitations of your review (e.g., "I could not verify the proofs in Appendix B in detail")
Security
- Security scanPassed, 95 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingStackPlanner2.0/skills/public/academic-paper-review/SKILL.md
Popularity
- Repository stars
- 442
- Repository forks
- 53
- 7-day trend score
- 30
Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/jiangxinke/harness-rl/academic-paper-review)