internet2/grouper/grouper-release

Grouper release

End-to-end steps to cut and record a Grouper container release. Companion skills do the heavy lifting for Jira and the wiki -- this one is the checklist and the release-notes-table contract.

• Jira writes (create version, resolve issues, set fixVersion): use grouper-jira.
• Wiki writes (edit the release-notes page): use grouper-wiki-edit (REST +

token, NEVER the MCP write path -- it corrupts storage-format XML).

Work a new release newest-first: the new row goes ABOVE the current top row.

Inputs to collect first

• New version, e.g. 7.3.2 (Grouper API version == container tag).
• Container base versions for the Versions cell: OS (e.g. Rocky 9.8), Tomcat

(e.g. 9.0.120), Java Corretto (e.g. 17.0.19.10.1).

• The built image sha256: digest (from the container build/push pipeline).

Not needed up front -- the row goes in with it blank (step 0).

• Release date (YYYY/MM/DD).
• Status for the new row: always RELEASED (see Status values). Promotion to

LATEST STABLE is a later step, confirmed with the user.

0-pre. Review open GitHub PRs

Before anything else, check for community PRs waiting on the release branch, so anything mergeable ships in this release (merge before tagging, step 2):

gh pr list --repo Internet2/grouper --state open --base GROUPER_7_BRANCH

Use the release line's branch for --base (e.g. GROUPER_6_BRANCH). For each PR, review the diff (/code-review <PR URL>) and give the user the findings. The user decides whether to merge or comment -- never merge, approve, or post on a PR yourself. A merged PR's GRP issue is then resolved/tagged with the others in steps 5-6. Also listed on the internal "Release steps for new container build" wiki page (GrIntDev, pageId 48794029).

0-review. Review every commit since the last release

Before the mirror and the tag, review every commit in the release range, grouped by GRP issue (commits without a GRP issue are reviewed too):

git log --no-merges --format='%h %an %s' --shortstat GROUPER_RELEASE_<prev>..origin/<BRANCH>

Review each issue's combined diff on four dimensions:

• Security: authorization (privilege checks, sysadmin-only paths), injection

(SQL/HQL, XSS -- unescaped JSP output, JS string building), secrets in logs or responses, new endpoints/servlets and their auth, unsafe defaults (a new feature should be off by default when it reaches outside Grouper, e.g. an AI provider).

• Functionality: does the code do what the issue says; edge cases (null,

blank, empty lists, paging), regressions in callers, transaction/cache handling, DB portability (postgres/oracle/mysql) for DDL and queries.

• Tests: is there a test that would fail without the change, and does it cover

the risky part. Note what is untested; do not write tests unasked.

• Documentation: new config keys exist in the *.base.properties with a

blank value and the default in the # {...} comment, matching the code default; new features have a wiki page or an update to an existing one; externalized text for new UI strings; upgrade instructions when operators must act.

Large issues (thousands of lines, e.g. a new feature) get their own reviewer; small ones can share one. Use parallel subagents, each returning findings with file:line, severity (blocker / should-fix / nit) and a concrete failure scenario. Verify any blocker yourself before reporting it.

Give the user one consolidated report, blockers first. The user decides what to fix before the tag; fixes go in as normal commits with their own (or the original) GRP issue, then re-check the commit-vs-Jira reconciliation (step 6).

0. Jira and wiki edits BEFORE the mirror

The mirror refresh (step 1) is committed and then tagged, so anything written to the wiki or Jira after that is not in this release's mirror. Do ALL the Jira and Confluence work that does not need the built image FIRST, then mirror, then tag:

1. DDL / upgrade tasks check (step 4).
2. Resolve the shipped GRP issues and set their fixVersion (steps 5, 6).
3. Feature/config doc pages the shipped changes affect (e.g. a config reference

page, a feature page, a security-issue page). Ask the user which pages; grep the wikiMirror for the config keys and features the commits touch.

1. v7 Upgrade instructions from v7 rows for anything operators must do or be

aware of (this sets the Upgrade instructions count in the row). New rows go on top. Every cell's data-highlight-colour follows the row's Importance: Important = pink #ffebe6, Medium important = yellow #fffae6, Not important = blue #deebff. Copying the row above keeps ITS colour, which is how the 7.7.0 Important row (GRP-7418) went in yellow; set it from the Importance instead, and re-check every row's colour after the edit.

1. A security-issue page for a fix in this release: mark the release as released

on the release date (timeline, affected-versions table, remediation text).

1. The release-notes row (step 8) with the sha256: left BLANK

(<p><span ...>sha256:</span></p>), and Rocky/Corretto taken from the registries (see "Rocky and Corretto without the image" in step 7).

After the image publishes, the only wiki edit left is pasting the digest into the row. That one edit is not mirrored until the next release, which is fine.

1. Refresh the grouper-docs mirrors FIRST

Do this BEFORE tagging, so the mirror refresh is part of the release commit rather than a commit that lands after the tag and leaves the branch ahead of it.

Run them in the checkout for the branch being released -- a v6 release refreshes v6's mirror. v7, v6 and v4 all carry both wikiMirror and issueMirror, so run both scripts on every branch; only sitemap.xml is v7-only (the v6/v4 wiki run does not produce one, and there is nothing to copy to i2midev6 for them).

Both mirrors are incremental, so this is cheap.

set -a && . ~/.secrets/grouper_confluence.env && set +a
cd grouper-misc/grouper-docs/wikiMirror  && python3 wikiToMarkdown.py
cd ../issueMirror                        && python3 jiraToMarkdown.py

Dependency: the scripts need beautifulsoup4, and it is often not installed (ModuleNotFoundError: No module named 'bs4'). Homebrew's python is externally-managed, so do not pip install into it -- use a throwaway venv:

python3 -m venv /tmp/wikivenv && /tmp/wikivenv/bin/pip install -q beautifulsoup4
/tmp/wikivenv/bin/python wikiToMarkdown.py

• The wiki run also regenerates wikiMirror/sitemap.xml (v7 only); copy it to

i2midev6:/var/www/html/sitemap.xml (see the sitemap section of wikiMirror/convertToMarkdownAiInstructions.md). Give the user the commands; they run them. i2midev6 serves grouperdemo.internet2.edu and is reached via login.internet2.edu (the maintainer's ssh alias is internet2_demo):

``bash scp /Users/mchyzer/git/grouper_v7/grouper-misc/grouper-docs/wikiMirror/sitemap.xml internet2_demo:/var/www/html/sitemap.xml # the maintainer's exact command (v7 checkout) curl -sI https://grouperdemo.internet2.edu/sitemap.xml # expect 200, xml content-type ``

The server file is owned by the maintainer's account (chowned once, mode 644), so plain scp overwrites it in place without sudo. If scp says permission denied, the file was deleted or re-owned: re-chown it (sudo chown <user> /var/www/html/sitemap.xml && sudo chmod 644 ...). Never mv a copy in from a home directory -- it keeps its home SELinux label and Apache returns 403; use sudo cp if a copy step is needed.

• jiraToMarkdown.py deliberately skips GRP-1 .. GRP-7143 -- the cloud

metadata for many of those is misaligned by the CSV import, so the existing files are the accurate record. See issueMirror/convertJiraToMarkdownAiInstructions.md.

• Review both diffs and commit separately (wiki: sync, issues: sync);

a human pushes.

What this run will NOT capture: only the sha256 digest in the release-notes row, since that needs the image, which needs the tag. Everything else was written in step 0 and IS captured. The digest gets mirrored by the NEXT release's run. Do not re-run the mirror after the tag just to pick it up, or you reintroduce the post-tag commit this ordering exists to avoid.

2. Push everything, THEN tag

The git tag is what Maven Central and the container build read. Tag a tree that is not fully pushed and the release itself is still correct -- the tag carries the commit -- but the BRANCH POINTER is left behind its own release tag, which is invisible unless someone checks for it explicitly.

This is easy to do by accident, because pushing the tag and pushing the branch are two separate operations and git push --tags does NOT push the branch. It happened on 6.4.0: GROUPER_RELEASE_6.4.0 was on origin pointing at the right commit while origin/GROUPER_6_BRANCH still pointed one commit back.

Before tagging, confirm the working tree is clean and nothing is unpushed:

git status --short                                    # expect no tracked changes
git log --oneline origin/<BRANCH>..<BRANCH>           # expect NO output

If that second command prints anything, push it first:

git push origin <BRANCH>

Only then tag, and push the tag:

git tag GROUPER_RELEASE_<version>
git push origin GROUPER_RELEASE_<version>

Then prove the tag and the pushed branch agree. Do not skip this -- it is the only check that catches the split, and it costs one command:

git fetch origin
git merge-base --is-ancestor GROUPER_RELEASE_<version>^{commit} origin/<BRANCH> \
  && echo "OK: tag is on the pushed branch" || echo "BAD: branch is behind the tag"

If it reports BAD, push the branch -- it is a plain fast-forward, since the branch head is an ancestor of the tagged commit. Never force-push or move the tag to "fix" this.

Remember the OTHER repos too. The container lives in a separate repo with a branch per version, and the same split applies there.

2a. Build the jars (grouper-build Jenkinsfile)

The jars are built and deployed to Maven Central by a separate repo, grouperContainer/grouper-build_v7 (v6: grouper-build_v6), with one branch per release like the container repo. The user creates the release branch and commits/pushes; the AI edits the Jenkinsfile on it. Check git branch --show-current shows the new version first.

Edit two things in Jenkinsfile:

• Version: def git_tag = '<version>' (e.g. '7.5.1'). It builds the

GROUPER_RELEASE_<version> tag, so the tag must already be pushed (step 2).

• Java: the JAVA_HOME='.../amazon-corretto-<ver>-linux-x64' line inside

environment {} -- set it to the current latest Corretto 17. Get it from the redirect the Jenkins JDK tool itself downloads:

``bash curl -sI https://corretto.aws/downloads/latest/amazon-corretto-17-x64-linux-jdk.tar.gz | grep -i '^location' ``

(.../resources/17.0.20.12.1/... -> 17.0.20.12.1). That line is informational; the mvn steps do not depend on it.

How the mvn steps get JAVA_HOME, and why it must not be resolved up front. The Corretto-JDK17 tool installs from the corretto "latest" URL, so withMaven(jdk: ...) re-unpacks it whenever Corretto releases, deleting the old nested amazon-corretto-* directory. On 7.5.1 the Jenkinsfile ran a find at node start, got 17.0.20.10.1, then the tool unpacked 17.0.20.12.1 over it and maven failed with The JAVA_HOME environment variable is not defined correctly. The fix (in the Jenkinsfile since 7.5.1): java_home is a single-quoted $(find ... -name "amazon-corretto-*" | sort -V | tail -1) string that each sh step expands AFTER the tool is installed. If you see that error again, check that nothing reintroduced an up-front sh(returnStdout: true) lookup.