valory-xyz/open-autonomy/bump-versions

Bump open-autonomy + open-aea (and upstream pins) in any downstream repo

End-to-end procedure for bumping a Valory downstream repo (mech, mech-interact, mech-client, mech-server, mech-predict, mech-agents-fun, trader, optimus, market-creator, market-resolver, meme-ooorr, kv-store, funds-manager, genai, etc.) to the latest released open-autonomy and open-aea. Repo-agnostic — keys off [project] dependencies and [tool.tomte] upstream_pins in pyproject.toml, not a hardcoded repo list.

The two frameworks that drive code changes are open-autonomy (OA) and open-aea (OAEA). Everything else listed in [tool.tomte] upstream_pins (e.g. valory-xyz/mech-interact, valory-xyz/genai, valory-xyz/mech) is bumped tag + third-party-hash only — never apply code rewrites for those.

The procedure assumes the modern uv + tox + PEP 621 `[project]` layout the downstream fleet now uses. Repos still on Poetry need the equivalent edits applied to [tool.poetry.dependencies] / poetry.lock — the phase structure is unchanged.

Phase 0 — Establish ground truth

Run these in parallel before doing anything else:

# 1. Confirm the current cwd is a Valory downstream repo (must have pyproject.toml + packages/packages.json)
test -f pyproject.toml && test -f packages/packages.json && echo "OK: looks like a downstream repo"
 
# 2. Current pins — match both layouts:
#    - PEP 621 / uv:  dependencies = ["open-autonomy[cli]==X", ...]   (leading quote)
#    - Poetry:        open-aea = {version = "==X", extras = [...]}     (bare key)
#    A PEP-621-only pattern silently no-ops on Poetry downstreams.
grep -E "^\s*(\"open-(autonomy|aea)|open-(autonomy|aea)[A-Za-z0-9_-]*\s*=)" pyproject.toml
 
# 3. Upstream pins listed for sync-only repos (read the whole [tool.tomte] block)
sed -n '/^\[tool\.tomte\]/,/^\[/p' pyproject.toml | grep -A100 "^upstream_pins"
 
# 4. Repo-local upgrading + history docs (some downstreams ship their own)
ls -1 HISTORY.md UPGRADING.md docs/upgrading.md 2>/dev/null

Record into working memory:

Phase 1 — Resolve target versions

If the user passed oa= / oaea=, use those. Otherwise hit GitHub's "latest release" endpoint — it returns whatever release is currently marked Latest in the UI, which (unlike gh release list | first) doesn't get fooled by an out-of-order backport release published after a newer one:

# Latest OA release (tag name includes the leading 'v'; strip if you need the bare version)
gh api repos/valory-xyz/open-autonomy/releases/latest --jq '.tag_name'
 
# Latest OAEA release
gh api repos/valory-xyz/open-aea/releases/latest --jq '.tag_name'

Set NEW_OA (e.g. 0.21.20) and NEW_OAEA (e.g. 2.2.3). If they equal the current pins, stop: prose-only guidance silently sails past a no-op run and produces a "touched lockfile / re-formatted packages.json" PR for zero version change. Gate the rest of the skill on a real diff:

if [[ "$NEW_OA" == "$CUR_OA" && "$NEW_OAEA" == "$CUR_OAEA" ]]; then
  echo "Already at latest (open-autonomy==$CUR_OA, open-aea==$CUR_OAEA). Nothing to bump."
  exit 0
fi

1.1 Resolve plugin coupling

Plugin versions are not independent — they track one framework or the other:

Verify the OA-side plugins are published for the target OA version (PyPI sometimes lags by a few minutes after a release):

# Anchor the version on a non-digit boundary so 0.21.2 doesn't false-match inside 0.21.20.
# Fail loudly if either plugin isn't on PyPI yet — silent grep-no-match would otherwise let
# the bump proceed and trip later with a confusing dependency-resolution error.
for pkg in open-aea-test-autonomy open-aea-helpers; do
  pip index versions "$pkg" 2>/dev/null \
    | grep -qE "(^|[^0-9])$NEW_OA([^0-9]|$)" \
    || { echo "ERROR: $pkg==$NEW_OA not yet on PyPI — hold the bump until the publish job completes"; exit 1; }
done

1.2 Verify OA at NEW_OA pins OAEA at NEW_OAEA

The bump only makes sense if the new OA itself pins the new OAEA. Pull OA's own pyproject.toml at v$NEW_OA and read the open-aea line(s) — OA still ships Poetry-style ([tool.poetry.dependencies]), so the pin shape is e.g. open-aea = {version = "==2.2.3", extras = ["all"]}. Materialize the fetch first so a network/auth failure is distinguishable from an empty grep result (the latter would be the trust-OA's-pin branch):

TMP=${TMP:-$(mktemp -d)}
gh api "repos/valory-xyz/open-autonomy/contents/pyproject.toml?ref=v$NEW_OA" \
  --jq '.content' > "$TMP/oa_pyproject.b64" \
  || { echo "ERROR: gh api failed for OA pyproject.toml@v$NEW_OA — check tag exists, gh auth, rate limit"; exit 1; }
base64 -d "$TMP/oa_pyproject.b64" | grep -E '^[[:space:]]*open-aea[[:space:]]*='

If OA v$NEW_OA pins a different OAEA version than the user asked for, trust OA's pin and override NEW_OAEA. A printed pin line that's only inspected by eye will scroll past in a Claude-driven run — extract, compare, and override programmatically so the mismatch can't reach Phase 3+:

OA_PINNED_OAEA=$(
  base64 -d "$TMP/oa_pyproject.b64" \
    | grep -E '^[[:space:]]*open-aea[[:space:]]*=' \
    | grep -oE '==[0-9]+\.[0-9]+\.[0-9]+' \
    | head -1 \
    | sed 's/^==//'
)
if [[ -z "$OA_PINNED_OAEA" ]]; then
  echo "ERROR: could not extract open-aea pin from OA pyproject.toml@v$NEW_OA"; exit 1
fi
if [[ "$OA_PINNED_OAEA" != "$NEW_OAEA" ]]; then
  echo "WARNING: OA v$NEW_OA pins open-aea==$OA_PINNED_OAEA; you asked for $NEW_OAEA — overriding to OA's pin."
  NEW_OAEA="$OA_PINNED_OAEA"
fi

A mismatch left in place would fail tox -e check-dependencies and autonomy packages sync hundreds of lines later, with a stack trace that doesn't name Phase 1.2 as the root cause.

1.3 Resolve upstream-repo tags (sync-only repos)

For each entry in [tool.tomte] upstream_pins, find the release tag that itself pins open-autonomy==$NEW_OA and open-aea==$NEW_OAEA:

# List recent releases
gh release list --repo valory-xyz/<repo> --limit 10
 
# Inspect the target tag's pyproject.toml for OA/OAEA pins
gh api "repos/valory-xyz/<repo>/contents/pyproject.toml?ref=<tag>" \
  --jq '.content' | base64 -d | grep -E "open-(autonomy|aea)"

Transitive upstream consistency: if trader is in upstream_pins AND mech-interact is in upstream_pins, the trader tag you pick must itself pin the same mech-interact version you picked. Otherwise autonomy packages sync will surface conflicting CIDs. Walk the dependency chain.

Release lag fallback: if an upstream has merged the OA/OAEA bump but not cut a release yet, you can pin to a commit SHA in pyproject.toml's [tool.uv.sources]:

[tool.uv.sources]
mech-interact = { git = "https://github.com/valory-xyz/mech-interact.git", rev = "<SHA>" }

Flag this to the user and update upstream_pins once the tag drops.

Phase 2 — Read upgrade docs and identify breaking changes

This is the only phase that drives code changes. Surface the diff between CUR_* and NEW_* for both frameworks, then apply the changes scoped to packages.json's dev map.

2.1 Fetch HISTORY + upgrading from upstream HEAD

HISTORY.md is authoritative — docs/upgrading.md is the user-facing distilled version. Read both: HISTORY for the full change set, upgrading for the call-to-action. Stage them in a fresh mktemp -d so parallel runs (or multiple users on a shared host) don't clobber one another.

Critical: each gh api … > file chain can fail silently — the redirect succeeds even if gh api errors, leaving a zero-byte file. Phase 2.2 then reads nothing, finds no breaking changes, and reports "no action required" — operator applies pin bumps without the code rewrites upgrading.md actually demanded. Guard every fetch:

set -euo pipefail
TMP=$(mktemp -d)
 
_fetch() {
  local repo="$1" path="$2" ref="$3" out="$4"
  gh api "repos/$repo/contents/$path?ref=$ref" --jq '.content' \
    | base64 -d > "$out"
  [[ -s "$out" ]] || { echo "ERROR: empty fetch — $repo:$path@$ref"; exit 1; }
}
 
# Open-autonomy — HISTORY.md and docs/upgrading.md
_fetch valory-xyz/open-autonomy HISTORY.md          "v$NEW_OA" "$TMP/oa_history.md"
_fetch valory-xyz/open-autonomy docs/upgrading.md   "v$NEW_OA" "$TMP/oa_upgrading.md"
 
# Open-aea — HISTORY.md and docs/upgrading.md
_fetch valory-xyz/open-aea      HISTORY.md          "v$NEW_OAEA" "$TMP/oaea_history.md"
_fetch valory-xyz/open-aea      docs/upgrading.md   "v$NEW_OAEA" "$TMP/oaea_upgrading.md"

Read only the sections between `CUR_OA` and `NEW_OA` in OA's docs, and between CUR_OAEA and NEW_OAEA in OAEA's docs. Both files are reverse-chronological — top of file is newest.

2.2 Classify each change

For every change between the two versions, tag it as one of:

Discipline: the upgrading.md sections are explicit about which changes downstream consumers must apply. Don't infer; if a section doesn't call for a change, don't make one. Conversely, every "Action required" / "Concrete upgrade steps" / "Breaking" callout is non-optional.

2.3 Where code changes apply (and where they don't)

Apply API/import rewrites only to files under paths in packages.json's "dev" map. Concretely:

# List dev package paths from packages.json
python3 -c "
import json
d = json.load(open('packages/packages.json'))
for k in d.get('dev', {}):
    # keys look like 'agent/valory/trader/0.1.0' → 'packages/valory/agents/trader'
    kind, author, name, _ = k.split('/')
    kind_dir = {'agent':'agents','skill':'skills','contract':'contracts','connection':'connections','protocol':'protocols','service':'services','custom':'customs'}[kind]
    print(f'packages/{author}/{kind_dir}/{name}')
"

Third-party packages (the "third_party" map) are never edited by hand — they get refreshed by autonomy packages sync --source in Phase 5.

Non-package code (top-level scripts/, tests/, app entrypoints) is also fair game for OA/OAEA API rewrites — only the packages/ tree is partitioned dev-vs-third-party.