mitodl/agent-kit/address-pr-feedback

Address PR Feedback

Turns a PR's review activity — and its CI/status checks — into fixes plus a clean, fully-resolved thread list and a green check bar. The core pattern is fetch → categorize → address → reply & resolve → verify, using scripts for the mechanical parts so the interesting work is reading feedback correctly and knowing when a decision belongs to a human instead of you.

See references/graphql-reference.md for the underlying queries/mutations, why plain first: N GraphQL calls silently truncate on long-running PRs, and why replying and resolving are separate mutations. See references/checks-reference.md for how the Checks API and third-party status checks differ, and how to re-run or diagnose each.

Recognizing the request

The trigger is almost always a short imperative naming a PR, sometimes bundled with other git operations:

• "Address the PR feedback [on #N / \<url\>]"
• "For any comment that is addressed, mark it as resolved"
• "Address PR feedback, resolving comments that have been handled"
• "Rebase, resolve the conflicts, and then address the PR feedback"
• "Address feedback across the set of open PRs" / "walk through the PR chain,

addressing feedback on the PRs first, then merge" (a stacked-PR batch)

• "Review the latest human-contributed feedback" / "there's more bot feedback

to evaluate" (a re-check after previously addressing a first round)

• "Fix the failing checks [on #N]" / "the CI is red, why?" / "address the

GitGuardian alert" / "prek is failing" / "pre-commit.ci is failing" — a checks-only request, not review-comment feedback; jump straight to Phase 1b — Checks below

• "Address the PR feedback" alone, with no mention of checks, still means

glance at the check bar (Phase 1b) — a PR with unresolved review comments and a failing required check is not addressed until both are handled

Resolve two things before running anything, asking if either is unclear:

1. Which PR(s)? A URL or #N in the message is unambiguous — use it.

When none is given, use this session as context before asking: if you opened or pushed to a PR earlier in this same conversation, that's almost always the one meant (dogfooding a skill you just built, continuing work you just pushed) — don't re-ask for something you already know. Otherwise fall back to gh pr view --json url,number for the current branch's tracked PR, and only ask the user if that's still ambiguous (no tracked PR, or reason to think the branch points at the wrong one). A request spanning multiple PRs (a stack, or "all my open PRs") means working through each one, one at a time — don't parallelize commits across PRs that depend on each other.

1. Report or act? This skill is almost always invoked to act (fix code,

reply, resolve) — that's what "address" means here, unlike `github-pr-triage` which defaults to report-only. If the user says "review the feedback" or "give me a summary" without "address"/"fix"/"resolve", treat it as read-only: categorize and report, but don't change code or touch thread state. A third answer sits between the two: "draft", "don't post", "leave pending", "show me first", "dry run", or "nothing goes up until I say" means draft-only mode. Fix locally, but push, post, and resolve nothing until the user approves.

Phase 1 — Fetch

./skills/process/address-pr-feedback/scripts/fetch-feedback.sh mitodl/agent-kit 116 /tmp/pr116-feedback.json

This is one call regardless of PR size — it follows GraphQL cursor pagination for review threads internally, so a PR with 200 threads across a dozen review rounds comes back complete, not truncated at the first page. By default it excludes already-resolved threads; add --include-resolved when you need the full history (e.g. re-checking a PR you partially addressed in an earlier session, or producing a final "here's everything, resolved and not" summary).

Output shape: {repo, pr, review_threads: [...], discussion_comments: [...], reviews: [...], pending_review: {...} | null}. Each review_threads[] entry carries the GraphQL id (pass this straight to resolve-thread.sh), isResolved, isOutdated, path/line, and its comments. Every comment, discussion comment, and review also carries author_type alongside author — GitHub's own actor type (User, Bot, Organization, Mannequin), normalized from GraphQL's __typename and REST's user.type so it reads the same regardless of which call produced the record. Classify authorship from that field, not from whether you recognize the login.

pending_review is the viewer's own unsubmitted review (null when there is none), with each comment's node id and a snapshot of its body. When it is set, the script also prints "You have a pending review with N comment(s)" to stderr. Pass that on to the user, and follow Editing a pending review comment before touching any of those comments.

Phase 1b — Checks

./skills/process/address-pr-feedback/scripts/fetch-checks.sh mitodl/agent-kit 116 /tmp/pr116-checks.json

Covers both GitHub Actions jobs (including the prek hook check) and third-party status checks (pre-commit.ci, GitGuardian, Sentry, CodeQL, and anything else posting to the PR's check bar) in one call. Excludes passing/skipped checks by default; add --include-passing for the full bar (e.g. a final "everything green" report).

Output shape: {repo, pr, checks: [...], action_run_logs: {<run_id>: <log text>}}. Each checks[] entry has name, bucket (pass/fail/pending/skipping/cancel), state, description, workflow, link, and a derived run_id when the check is a GitHub Actions job. For every failing check with a run_id, action_run_logs holds that run's failed-step output (deduped — several failing jobs from the same workflow run share one log entry), truncated to the last ~20k characters, so you can usually diagnose without a separate gh run view call. Checks with run_id: null (pre-commit.ci, GitGuardian, Sentry, a legacy commit-status check, etc.) have no fetchable log here — their link points at the external service; fetch it directly if it's a public page, otherwise summarize from description and ask the user if more detail is needed — rather than guessing at the failure from the name alone.

A pending check isn't a failure — don't "fix" something that's still running. If everything is pass/pending and the user asked to address checks specifically, say so and stop; there's nothing to act on yet beyond noting what's still in flight.

Phase 2 — Categorize

Group by reviewer and tag each item with a priority, inheriting the reviewer's own label when it supplies one (bot reviewers like gemini-code-assist and copilot-pull-request-reviewer usually embed a severity in the comment body or as a badge) — otherwise infer from substance:

1. **gemini-code-assist** (medium): transport type mismatch in agent-config.toml
2. **copilot-pull-request-reviewer** (nitpick): variable naming in fetch loop
3. **human — tmacey** (substantive): reconsider the retry backoff strategy

isOutdated: true does not mean "safe to skip" — the line moved but the underlying concern may still apply. Read the current code at that path before deciding a stale-looking thread is moot.

Bot review-state fields (APPROVED/COMMENTED/CHANGES_REQUESTED in reviews[]) tell you formality, not substance — most bots (Copilot, Gemini) leave everything in COMMENTED state with the real content in the thread comments, which is why `github-pr-triage` reads unresolved threads rather than review states.

A human reviewer's comment gets more deliberate handling than a bot's, even when the fix looks equally obvious. A bot flags a pattern; a human colleague is making a judgment call, and a quick autonomous fix-and-resolve can read as brushing off their input. For human-authored threads: don't auto-resolve on your own judgment alone — bring the proposed fix (or your reasoning for declining) to the operator before pushing and resolving, and use their framing in the reply rather than your paraphrase. Bot-authored threads (Copilot, Gemini, CodeQL, etc.) don't need that same checkpoint — address, reply, and resolve those directly per the rest of this skill. In draft-only mode, bot threads wait for approval too.

Decide which branch applies from author_type (Phase 1's output), never from whether the login looks like a bot — an unfamiliar GitHub App or a newly-added reviewer has no recognizable name, and guessing wrong applies the wrong handling to a colleague. Bot (and Mannequin, an imported placeholder identity) take the bot branch; Organization is rare on review threads and takes the human branch.

User is the one value that doesn't settle it: a service account driving automation through a normal user token reports User exactly like a person does, because GitHub has no field that distinguishes them. When a User thread reads like machine output — templated phrasing, a severity badge, an identical comment repeated across files — say that's your read and ask, rather than silently taking the bot branch on a thread a colleague wrote.

Failing checks split into three kinds that get handled differently in Phase 3 — tag each one on the way in: