yugabyte/yb-voyager/pr-review-sweep

PR Review Sweep

Reviews open PRs with REVIEW.md and posts the findings with the post-pr-review skill, covering PRs that need a review — never reviewed, or carrying new commits since the last AI review. Designed to run unattended (e.g. from a daily scheduled routine), so it never pauses for confirmation.

This skill does not duplicate the review logic — it selects PRs, then delegates each one to a reviewer subagent and a poster subagent. The reviewer applies REVIEW.md (repo root, also what the branch-review skill points to). The poster applies post-pr-review with one documented override. REVIEW.md and ai/skills/post-pr-review/SKILL.md are the source of truth for how a review is produced and posted.

Defaults

Modes

• Default mode (no args): select PRs per the filters above.
• Explicit-PR mode (/pr-review-sweep 3512 3520): review exactly the given PR numbers. All filters are bypassed — recency, draft, bot, size, and the already-reviewed skip (the user asked for these PRs, so re-review even if an [AI] review exists).
• --days N / --limit N adjust the defaults in default mode.

Workflow

Copy this checklist and track progress in your response:

- [ ] Step 0: Prerequisites (gh auth, repo resolution)
- [ ] Step 1: Build the candidate list
- [ ] Step 2: Decide per PR — full review, incremental re-review, or skip
- [ ] Step 3: Apply the size cutoff and the per-run cap
- [ ] Step 4: Process each PR sequentially via two subagents (reviewer, then poster)
- [ ] Step 5: Print the run summary table

Step 0: Prerequisites

gh auth status          # must be authenticated with repo scope
OWNER_REPO=$(gh repo view --json nameWithOwner -q .nameWithOwner)
REPO_ROOT=$(git rev-parse --show-toplevel)

If gh is missing or unauthenticated, stop and report — do not attempt anonymous API calls. The repository is always derived from origin; never hardcode it.

Step 1: Build the candidate list

Skip this step in explicit-PR mode (the user-supplied numbers are the list — just confirm each exists and is open with gh pr view <N> --json state; warn and skip any that aren't open).

DAYS=7   # or --days override
gh pr list --state open --limit 100 \
  --json number,title,author,updatedAt,isDraft,baseRefName,additions,deletions \
  --jq "[ .[]
        | select(.isDraft | not)
        | select((.author.is_bot // false) | not)
        | select((.updatedAt | fromdateiso8601) >= (now - ${DAYS}*86400)) ]
        | sort_by(.updatedAt)"

Notes:

• sort_by(.updatedAt) ascending → oldest-updated PRs are processed first, so nothing starves under the per-run cap.
• The fromdateiso8601-based filter is portable (no date -v / date -d divergence between macOS and Linux).

Step 2: Decide per PR — full review, incremental re-review, or skip

Gather the PR's AI-review history:

# Latest AI review: submitted_at + the commit it reviewed
gh api --paginate "repos/$OWNER_REPO/pulls/$N/reviews" \
  --jq '[ .[] | select(.body // "" | startswith("[AI]")) ] | max_by(.submitted_at) | {submitted_at, commit_id}'
 
# Latest AI marker comment (zero-findings runs post these; no commit_id)
gh api --paginate "repos/$OWNER_REPO/issues/$N/comments" \
  --jq '[ .[] | select(.body // "" | startswith("[AI]")) ] | max_by(.created_at) | .created_at'
 
# Current head
gh pr view $N --json headRefOid --jq .headRefOid

Decision (take the most recent of the review/marker timestamps as LAST_AI):

• No AI review or marker → full review.
• No new commits since `LAST_AI` (head commit == last AI-reviewed commit_id, or — when the last AI activity was a marker with no commit_id — the head commit's committer date <= LAST_AI) → skipped (already reviewed, no new commits).
• New commits, and `LAST_AI` more than 24h ago → incremental re-review of the range <last AI commit_id>..head (see Step 4). If the last AI activity was only a marker (no commit_id) or the old commit is gone (force-push), do a full re-review instead.
• New commits, but `LAST_AI` within the last 24h → skipped (re-reviewed recently) — it will be picked up by a later run.

In explicit-PR mode this logic is bypassed entirely (always a full review).

Step 3: Size cutoff and cap

• If additions + deletions > 5000, record the PR as skipped (too large: <X> lines) and post nothing — the PR stays unmarked so a human (or an explicit-PR run) can still review it.
• Take the first 5 remaining PRs (oldest-updated first). Record the rest as deferred (over per-run cap) — they'll be picked up by the next run.

Step 4: Process each PR sequentially via two subagents

Process PRs strictly one at a time (never in parallel — sequential runs keep gh rate usage and repo state predictable). For each PR spawn two subagents in sequence: a reviewer whose only job is the review, then a poster whose only job is publishing the findings. Do not merge them into one agent — a single agent juggling setup, review, and posting produces shallow reviews.

4a. Reviewer subagent — prompt of this shape:

Review the code changes for GitHub PR #<N> ("<title>") in <OWNER_REPO>.
Your ONLY job is the review — do not post anything. Work non-interactively.
 
Repo root: <REPO_ROOT>. Base branch: <BASE>.
 
1. Set up an ephemeral worktree of the PR head (never touch the main checkout):
     cd <REPO_ROOT>
     git fetch origin "refs/pull/<N>/head"
     git worktree add --detach /tmp/pr-review-sweep-<N> FETCH_HEAD
     git fetch origin <BASE>
   Then work inside /tmp/pr-review-sweep-<N>.
 
2. Read <REPO_ROOT>/REVIEW.md and apply it fully and exhaustively to this
   worktree, with BASE=origin/<BASE> (the PR's base branch, not main), i.e.
   MERGE_BASE=$(git merge-base "origin/<BASE>" HEAD).
   HEAD is detached here; use "PR #<N>" as the branch name in the summary.
   Depth matters more than speed: read the full changed files, run the
   REVIEW.md's mandatory mechanical checks, and apply every lens. Do not stop at the
   first few findings.
 
[Incremental mode only — include when Step 2 chose incremental re-review:]
3. This PR was already reviewed at commit <LAST_COMMIT>. Focus the review on
   the new range: git diff <LAST_COMMIT>..HEAD. Also re-check whether the
   previously posted [AI] findings (fetch them via
   gh api repos/<OWNER_REPO>/pulls/<N>/comments) were addressed by the new
   commits; do not re-report a finding that is already posted and still
   anchored to unchanged code.
 
4. Do NOT remove the worktree — the posting step needs it to verify
   comment anchors against the PR head. Cleanup happens after posting.
 
5. Your final message is the complete findings report (every Critical,
   Warning, Suggestion, and Question with file:line), plus the change
   summary. This report is the input to a posting step — do not truncate it.

4b. Poster subagent — give it the reviewer's findings report verbatim and a prompt of this shape:

Post this review to GitHub PR #<N> in <OWNER_REPO>. Work non-interactively.
 
The PR head is checked out in the worktree /tmp/pr-review-sweep-<N>; run
the anchor-verification `git show HEAD:...` checks from inside it so line
numbers are verified against the PR head, not the main checkout.
 
Read <REPO_ROOT>/ai/skills/post-pr-review/SKILL.md and apply it, with ONE
override: skip Step 6 (user confirmation) — post immediately after building
and validating the payload. Zero findings → post the "no findings" marker
comment exactly as that skill's Step 3 specifies.
All other rules in that skill (COMMENT event, [AI][<Severity>] prefixes,
JSON via --input file, anchor verification) apply unchanged. If a 422 says
a comment line is not part of the diff, drop or re-anchor that finding and
retry once.
 
Cleanup (always, even on failure):
  cd <REPO_ROOT>
  git worktree remove --force /tmp/pr-review-sweep-<N>
  rm -f /tmp/pr-review-<N>.json /tmp/pr-review-<N>-response.json
 
Findings report:
<REVIEWER REPORT>
 
Your final message must be exactly one of:
  POSTED <review html_url> (<c> critical, <w> warning, <s> suggestion)
  MARKER (no findings)
  FAILED <one-line reason>

Failure policy: a FAILED PR never aborts the sweep — record it and continue with the next PR. Do not retry within the run; a failed PR stays unmarked, so the next scheduled run retries it naturally. If the reviewer fails (so the poster never runs), the sweep itself removes /tmp/pr-review-sweep-<N> before moving on.

Step 5: Run summary

End with a table covering every PR considered, including filtered ones:

## PR Review Sweep — <date>
 
| PR | Title | Outcome |
|----|-------|---------|
| #3512 | Fix sequence restore ordering | posted — 1 critical, 2 warnings, 3 suggestions (<review-url>) |
| #3515 | Retry snapshot batches        | re-reviewed (incremental) — 1 warning (<review-url>) |
| #3520 | Refactor import batching      | marker — no findings |
| #3518 | Add Oracle RAC support        | skipped (too large: 8 412 lines) |
| #3501 | Bump deps                     | skipped (already reviewed, no new commits) |
| #3499 | Live migration docs           | failed (422: line not in diff after retry) |

Plus one line of totals: N reviewed, R re-reviewed, M markers, K skipped, J failed, D deferred.

Anti-patterns

• Never pause for user confirmation — this skill's contract is unattended operation. The safety properties come from post-pr-review's rules (non-blocking COMMENT, [AI][<Severity>] prefixes), not from a human gate.
• Never collapse the reviewer and poster into one subagent. A single context doing setup + review + posting has produced shallow reviews before; the dedicated reviewer is what protects depth.
• Never modify or check out branches in the main working copy. All PR checkouts go through ephemeral /tmp worktrees.
• Never compare against hardcoded `main`. Always diff against the PR's baseRefName.
• Never mark a PR as reviewed without posting something. The marker comment is what makes the sweep idempotent — skipping it makes the PR get fully re-reviewed every day.
• Never run PR subagents in parallel.
• Never let one PR's failure abort the sweep.