evalstate/fast-agent/pr-writing-review

PR Writing Review

Extract editorial feedback from GitHub PRs to learn from review improvements.

Prerequisites

• GitHub CLI: gh installed
• Authenticated `gh` session: gh auth status should show you’re logged in
• For private repos, your token needs appropriate scopes (typically repo).
• Python: 3.12+
• uv (recommended): https://github.com/astral-sh/uv

Division of Labor

Quick Start

 
> **All paths are relative to the directory containing this SKILL.md file.**
> Before running any script, first `cd` to that directory or use the full path.
 
# Get suggestions and feedback
uv run scripts/extract_pr_reviews.py <pr_url>
 
# Get full first→final comparison for deep analysis
uv run scripts/extract_pr_reviews.py <pr_url> --diff
 
# Same as above, but cap each FIRST/FINAL dump to 2k chars for LLM prompting
uv run scripts/extract_pr_reviews.py <pr_url> --diff --max-file-chars 2000

Workflow

Step 1: Extract with --diff

uv run scripts/extract_pr_reviews.py https://github.com/org/repo/pull/123 --diff

This outputs:

1. Explicit Suggestions — exact before/after text from suggestion blocks (supports multiple suggestion blocks per comment)
2. Reviewer Feedback — plain text comments (the "why" behind changes)
3. File Evolution — first draft and final version of each text file

Tip: add --max-file-chars 2000 to keep each FIRST/FINAL dump lightweight, or pair --diff with --no-files if you only need the suggestion/feedback summaries.

Step 2: Analyze the Output

With the script output, perform this analysis:

A. Catalog the Explicit Suggestions

Create a table of mechanical fixes:

B. Map Feedback to Changes

For each reviewer feedback comment:

1. Find the relevant section in FIRST DRAFT
2. Find the same section in FINAL VERSION
3. Document what changed and why

Example:

Feedback: "would be nice to end more enthusiastically"

First draft: "...it's simple to add new tools to Claude and use them straight away."

Final: "...Let us know what you find and create in the comments below!"

Lesson: End blog posts with a call-to-action

C. Paragraph-by-Paragraph Comparison

Compare FIRST DRAFT to FINAL VERSION section by section:

• What was added?
• What was removed?
• What was reworded?
• What structural changes were made?

D. Synthesize Style Patterns

Group findings into categories:

Script Options

📁 All paths are relative to the directory containing this SKILL.md file.

Input formats: pass either a full PR URL, owner/repo PR_NUMBER, or owner repo PR_NUMBER.

Output Structure

Default Output

• Writing Suggestions: Grouped by reviewer, shows original→suggested text (fenced blocks) along with any reviewer note and a permalink back to GitHub
• Reviewer Feedback: Plain comments without code suggestions, each tagged with its GitHub link

With --diff

• Explicit Suggestions: Compact before/after pairs, reviewer notes, and GitHub permalinks in one place
• Reviewer Feedback: Numbered list of requests (same as default view)
• File Evolution: FIRST DRAFT and FINAL VERSION for each .md/.txt/.rst/.mdx file; add --max-file-chars to truncate each block with a visible ...[truncated X chars] indicator

Handling File Renames

The script traces files through renames by:

1. Checking each commit for rename operations
2. Building a path history (e.g., claudeimages.md → claude-images.md → claude-and-mcp.md)
3. Fetching content using the correct path for each commit

Example Analysis Output

After running the script and performing LLM analysis, produce a summary like:

## Style Lessons from PR #123
 
### Mechanical Fixes
 
-Fix grammar: "Its" → "It's" (contraction)
-Lowercase generic terms: "Image Generation" → "image generation"
-Remove filler: "the output quality of" → "the quality of"
 
### Reviewer-Driven Changes
 
-**"end more enthusiastically"** → Added call-to-action in conclusion
-**"emphasize these are SoTA"** → Changed "latest" to "state-of-the-art"
-**"add blurb about MCP Server"** → Added explanatory paragraph
 
### Structural Improvements
 
-Added transition sentence between sections
-Simplified setup instructions (3 sentences → 1)
-Added new bullet point for model flexibility

Limitations

• Only extracts inline PR review comments (not issue comments or the PR description)
• Extremely long files can still be heavy; when that happens, lower --max-file-chars or pass --no-files to keep outputs prompt-friendly

{{currentDate}} {{env}}