yugabyte/yb-voyager/post-pr-review

Post PR Review

Posts an AI-generated code review to a GitHub pull request as a single `COMMENT` review with per-finding line-anchored comments. Designed to be a safe, clearly-marked way to surface AI review findings to a PR without requesting changes or approving.

Non-negotiable rules

1. Event type is always `COMMENT`. Never REQUEST_CHANGES, never APPROVE. The human reviewer decides whether to block the PR.
2. Every inline comment body starts with `[AI]` followed by a severity tag — [AI][Critical] , [AI][Warning] , or [AI][Suggestion] . The overall review body also begins with [AI].
3. Critical, Warning, and Suggestion findings are posted. Each is normalized to one of those three tags (Step 3). Questions and praise/meta commentary are dropped — questions stay in the in-chat review for the user, but posting them would make PR reviews too verbose. If there are zero postable findings, post a "no findings" marker comment (Step 3) — a clean review is information too.
4. Each inline comment must be line-anchored to a line in the PR diff. Use side: "RIGHT" and the line number from the current head of the PR branch. Un-anchorable findings (e.g. "the package has no tests") must be anchored to a sensible representative line in a new/changed file, not posted as a top-level comment.
5. Post via `gh api` with a JSON `--input` file, not by stringifying the payload inline. JSON escaping in shell is how reviews get corrupted.

Prerequisites

command -v gh            # gh CLI must be installed
gh auth status           # must be logged in; token scope `repo` is sufficient

If gh is missing or unauthenticated, stop and ask the user to install and run gh auth login — do not attempt to post via raw curl.

Workflow

Copy this checklist and track progress in your response:

- [ ] Step 1: Identify the PR (number, owner, repo)
- [ ] Step 2: Collect findings from the conversation
- [ ] Step 3: Normalize severities (map to Critical / Warning / Suggestion; drop Questions and praise)
- [ ] Step 4: Verify each finding has a file path and anchor line
- [ ] Step 5: Build the review JSON payload
- [ ] Step 6: Show the user a summary and confirm
- [ ] Step 7: POST the review
- [ ] Step 8: Verify response and surface the review URL
- [ ] Step 9: Clean up temp file

Step 1: Identify the PR

Resolve in this order:

1. If the user provided a PR number or URL, use it.
2. Else, look up the PR for the current branch:

``bash BRANCH=$(git rev-parse --abbrev-ref HEAD) REMOTE_URL=$(git config --get remote.origin.url) # Parse owner/repo from REMOTE_URL, then: gh pr list --head "$BRANCH" --state open --json number,url,title,headRefName --limit 5 ``

1. If zero or multiple matches, ask the user which PR.

Record owner, repo, and pull_number — you will need them in Step 7.

Step 2: Collect findings from the conversation

Findings almost always come from a preceding branch-review pass in the same chat. Extract them as structured items. Do not re-run the review; reuse what was already produced.

For each finding, capture:

• Severity: Critical | Warning | Suggestion | anything else
• File path (repo-relative, forward slashes)
• Line number (must be a line in the PR diff's RIGHT side)
• Title (one-line summary — becomes the first line of the comment body)
• Body (the multi-line explanation)

Step 3: Normalize severities

Map each finding (case-insensitive) onto one of the three posted tags:

• Critical ← Critical, Must fix, Blocker, Bug
• Warning ← Warning, Should fix
• Suggestion ← Suggestion, Nice to have, Good to have, Optional, Style, Nit, Polish

Drop (silently): Question / Needs author input (kept in the in-chat review only — posting them makes reviews too verbose), Praise / Positive, and anything that is commentary rather than an actionable finding.

If zero findings remain, do not build a review payload. Instead (after the Step 6 confirmation) post a single conversation comment so the PR carries a visible "reviewed, nothing found" record:

gh api --method POST "repos/<owner>/<repo>/issues/<pull_number>/comments" \
  -f body='[AI] Automated review — no findings.'

Step 4: Verify each finding has a file path and anchor line

For each remaining finding:

1. Confirm the path exists in the PR diff:

``bash gh pr diff <pull_number> --name-only | grep -F -x "<path>" ``

1. Confirm the line number is reachable on side: "RIGHT" (i.e. it exists in the PR head version of that file, not only in base). Read the file at the branch HEAD to confirm the line is present:

``bash git show HEAD:"<path>" | sed -n '<line>p' ``

1. If a finding is about the absence of something (e.g. "no unit tests for this package"), anchor it to a representative line inside a new/changed file in the same package (e.g. the func PrintBanner( line). Do not fabricate line numbers.

If any finding cannot be anchored, either (a) re-anchor it to a reasonable line or (b) drop it and note the drop in the final summary you give the user.

Step 5: Build the review JSON payload

Write to a temp file under /tmp. Never inline the JSON into the shell command.

Payload shape:

{
  "event": "COMMENT",
  "body": "[AI] Automated review — N findings (C critical, W warning, S suggestion). See inline comments.",
  "comments": [
    {
      "path": "path/to/file.go",
      "line": 123,
      "side": "RIGHT",
      "body": "[AI][Warning] **One-line title.**\n\nMulti-line explanation...\n\nSuggested fix: ..."
    }
  ]
}

Rules for comment bodies:

• Prefix: begins with [AI][<Severity>] exactly once, where <Severity> is one of Critical, Warning, Suggestion. Never [ai], never **[AI]**, never (AI).
• Title: the one-line finding title should be bolded on the first line after the prefix, e.g. [AI][Warning] **Double-close race in stopSpinnerLocked.**
• Body: separate from the title by a blank line. May include code blocks, file paths in backticks, and quoted AGENTS.md standards when they motivated the finding.
• Keep it self-contained: the reader on GitHub may not have the chat context. Include the "why" and a concrete suggested fix.
• No emojis unless the user explicitly asked the branch-review to use them.

Event-level rules:

• "event": "COMMENT" — hard requirement.
• The overall body is a short, factual one-liner that counts the findings. It starts with [AI] so the review card on GitHub is identifiable at a glance.

Write the file:

cat > /tmp/pr-review-<pull_number>.json <<'JSON'
{ ...payload... }
JSON

Or, preferred when building programmatically, write via the Write tool.

Step 6: Show the user a summary and confirm

Before posting, display a compact table so the user can sanity-check:

PR: yugabyte/yb-voyager#3497
Event: COMMENT (non-blocking)
Review body: [AI] Automated review — 7 findings (0 critical, 6 warning, 1 suggestion)...
 
| # | Severity   | File                                         | Line | Title                                                |
|---|------------|----------------------------------------------|-----:|------------------------------------------------------|
| 1 | Warning    | yb-voyager/cmd/assessMigrationCommand.go     |  330 | Verbose output undermines Preflight block            |
| 2 | Suggestion | yb-voyager/src/ux/progress.go                |  194 | Latent double-close race in stopSpinnerLocked        |
...

Ask the user if they want any edits before posting. If they say "go" / "post" / "send", continue.

Step 7: POST the review

gh api --method POST \
  /repos/<owner>/<repo>/pulls/<pull_number>/reviews \
  --input /tmp/pr-review-<pull_number>.json \
  > /tmp/pr-review-<pull_number>-response.json 2>&1
echo "exit=$?"

Do not use gh pr review — its flags don't support per-line comments cleanly. Use the raw API.

Step 8: Verify and surface the URL

Parse the response:

python3 -c "
import json, sys
d = json.load(open(sys.argv[1]))
print('review_id:', d.get('id'))
print('state:', d.get('state'))
print('html_url:', d.get('html_url'))
print('n_comments:', len(d.get('comments_url','')) and 'see review')
" /tmp/pr-review-<pull_number>-response.json

If state is not COMMENTED, show the raw response to the user and stop. Common failure modes:

• 422 Unprocessable Entity with "Pull request review thread line must be part of the diff" — a finding anchored to a line outside the diff. Drop or re-anchor that finding and retry.
• 401 Bad credentials — token expired. Tell the user to re-auth.
• 403 Resource not accessible — token lacks repo scope for the target repo.

On success, give the user a short confirmation with the html_url.

Step 9: Clean up

rm -f /tmp/pr-review-<pull_number>.json /tmp/pr-review-<pull_number>-response.json

Anti-patterns

• Never stringify JSON in the shell. Always --input <file>. Shell-quoting bugs have silently dropped inline comments before.
• Never use `REQUEST_CHANGES` or `APPROVE`. The agent does not have the standing to block or approve a human's PR.
• Never omit the `[AI][<Severity>]` prefix, even when the comment quotes an AGENTS.md standard or references another comment. The [AI] prefix is the contract with the human reviewers.
• Never re-run `branch-review` to get findings. If the conversation already produced findings, reuse them. Re-running wastes tokens and produces noise.
• Never invent line numbers. If you can't anchor a finding, drop it or ask the user where to anchor it.

Example: minimal happy path

# Input from prior branch-review: 6 Warnings, 8 Suggestions, PR #3497 on yugabyte/yb-voyager.
# Step 3: severities normalized → 14 findings to post.
 
# Step 5: write payload
#   (done via Write tool to /tmp/pr-review-3497.json)
 
# Step 7: post
gh api --method POST \
  /repos/yugabyte/yb-voyager/pulls/3497/reviews \
  --input /tmp/pr-review-3497.json \
  > /tmp/pr-review-3497-response.json
 
# Step 8: verify
#   → state: "COMMENTED"
#   → html_url: https://github.com/yugabyte/yb-voyager/pull/3497#pullrequestreview-...
 
# Step 9: cleanup
rm -f /tmp/pr-review-3497.json /tmp/pr-review-3497-response.json

Example comment body

[AI][Warning] **Latent double-close race in `stopSpinnerLocked`.**
 
This method temporarily releases `t.mu` while waiting on `spinnerDone`, then
re-acquires it to zero out `spinnerStop`. If any other goroutine grabs `t.mu`
during that release window, it will see a still-non-nil `spinnerStop`, call
`close(t.spinnerStop)` on an already-closed channel, and panic.
 
Today the tracker is driven by a single external goroutine, so this doesn't
trigger in practice. But the mutex nominally protects exactly this invariant.
 
Suggested fix: zero `t.spinnerStop` (and `spinnerDone`) **before** the
`t.mu.Unlock()`, then wait on the (local-variable captured) `spinnerDone`
channel without the lock.

Scope limits

This skill is only for posting reviews, not producing them. If the user hasn't done a review yet, suggest they run the branch-review skill first, then invoke this one.