google/ax/substrate
Claude Code and agent skill by google/ax
- vendor
Checking that the SKILL.md file is still in the repository.
How AX depends on Agent Substrate (modules, address contract, mocks, version bumps, E2E pin). Load when touching internal/substrate, bumping Substrate, or triaging make test-e2e.
Install
npx vskill@latest install google/ax/substrateSKILL.md
The same SKILL.md was published earlier as google/ax/substrate.
Agent Substrate in AX
AX schedules every Task as a sandboxed actor on Agent Substrate. AX owns the declarative layer (Task/Workspace/Model, reconciler, CLI); Substrate owns isolation, checkpoint/restore, worker placement and in-cluster routing. Facts below were verified against main on 2026-11-06 — re-check go.mod and hack/setup-e2e-kind.sh if they look stale.
1. Module map
| Module | Pinned | Provides | Imported by | |
|---|---|---|---|---|
go.ate.dev/substrate | v1.0.0 | Control API protos api/ateapi/v1 (atespaces, actor templates, actors, tags, snapshot fidelity, suspend/resume, egress rules) and pkg/postgressetup | internal/substrate, internal/substrate/substratetest, internal/install/substrate, internal/server, internal/controller, internal/apivalidation, test/e2e, pkg/apis/v1alpha1 (its .proto files import ateapi.proto) | |
github.com/agent-substrate/env | v0.0.11-0.20260912052224-4468a200b170 (pseudo-version) | Guest services served inside the sandbox (process exec + file system gRPC) and proto/ateenv/v1alpha | internal/guest, internal/metadata |
There is no replace directive for either module. Local hacking against a Substrate checkout goes through SUBSTRATE_DIR in E2E (section 5).
2. The Control API wrapper: internal/substrate.Client
All Control API traffic goes through this package. It encapsulates:
- Endpoint defaults —
api.ate-system.svc.cluster.local:443; authority
and TLS ServerName default to api.ate-system.svc when the target contains that string.
- Auth — bearer token from
SUBSTRATE_TOKEN_FILE(in-cluster: projected SA
token with audience api.ate-system.svc, see deploy/ax-server.yaml), CA from SUBSTRATE_CA_FILE.
- Retry —
IsTransientError(Unavailable,DeadlineExceeded,Aborted) +callWithRetry(retryOnCodes). - Helpers the reconcilers rely on —
EnsureAtespace,EnsureActorTemplate,
EnsureActorTemplateWithImage, BuildActorTemplate, EnsureActor, EnsureActorFromTag, GetActorEgressPolicy, EnsureActorEgressPolicy, CreateTag, GetTag, ListTags, DeleteTag, SuspendActor, SuspendActorAndWaitForSnapshot, ResumeActor, DeleteActor, WorkerIP, ResourceLimits.
Where a new Control API call lives today
internal/substrate/client.gois the only non-test file that calls
ateapipb.NewControlClient; every existing RPC is a method on *Client, wrapped in callWithRetry (codes.Unavailable plus optional codes such as codes.Aborted).
- Each method has a table-driven case in
client_test.go(mockControlClient). - Callers in
internal/controllerexercise it throughmockControlServer
(reconciler_test.go) or internal/substrate/substratetest (workspace_test.go); internal/server and cmd/ax tests also use internal/substrate/substratetest (section 4).
3. Address contract
| Value | Occurrences | |
|---|---|---|
api.ate-system.svc.cluster.local:443 + authority api.ate-system.svc | cmd/ax-server/main.go (--substrate-endpoint / --substrate-authority flag defaults), internal/substrate/client.go (3×), deploy/ax-server.yaml, test/e2e/manifests/ax-server.yaml, test/e2e/harness_test.go | |
atenet-router.ate-system.svc[.cluster.local] | internal/controller/reconciler.go (defaultRouterServerName), internal/tunnel/router_tls.go (routerServerName), cmd/ax/main.go (ssh port-forward path), internal/guest/client.go (doc comment), deploy/ax-server.yaml, test/e2e/manifests/ax-server.yaml, docs/networking.md | |
namespace ate-system | all of the above + test/e2e/harness_test.go (ateSystemNamespace), test/e2e/manifests/credential-policy.yaml, internal/install/axdeploy/authz.go, internal/install/cluster/kind.go, internal/install/substrate/bootstrap.go, internal/install/substrate/pki.go, hack/sync-substrate-manifests.sh (and generated internal/install/substrate/manifests/*.yaml) | |
pod labels ate.dev/worker-pool (worker pods) and app: atenet-egress (egress gateway) | the ax-server NetworkPolicy in deploy/ax-server.yaml and test/e2e/manifests/ax-server.yaml, which blocks both from :8080/:9090 and allows only app: atenet-egress (in ate-system) to reach :8081 (ax-server-runner) | |
opentelemetry-collector.otel-system.svc:4317 (the collector Substrate's kind overlay deploys, manifests/ate-install/kind/ate-otel-config.yaml in Substrate) | test/e2e/manifests/ax-server.yaml (ax-otel-config ConfigMap). An unreachable address does not fail E2E; ax-server only logs OTLP export errors | |
kind-registry:5000, the registry container that serves the images of a loopback registry (localhost:5001/...) to an ax local cluster | Only atelet's --localhost-registry-replacement in internal/install/substrate/manifests/05-ate-system.yaml (generated; actor images). LocalRegistryEndpoint (internal/install/substrate/bootstrap.go) reads it from there, and cmd/ax/install_cmd.go passes it as cluster.UpOptions.LocalRegistry to the Kind provider (internal/install/cluster/registry.go), which writes the containerd mirror for pod images such as ax-server. No Go file names the registry; docs/local_installer.md does |
Why it matters: tasks never get a Service/Ingress. Every request to a task is proxied by the atenet router, which picks the target from the ate-target-actor header and resumes a suspended actor on demand (docs/networking.md). ax ssh port-forwards to svc/atenet-router on 443 (mTLS) and falls back to 80.
4. Testing against Substrate
Unit tests: in-process mock, no cluster
Canonical examples: internal/controller/reconciler_test.go (reconciler state transitions; the largest test file in the repo). internal/server/server_test.go and cmd/ax/main_test.go run a real TaskReconciler against internal/substrate/substratetest, an in-memory fake of the same API (atespaces, actor templates, actors, and tags). internal/substrate/client_test.go uses a client-side mockControlClient instead. No mocking library is in go.mod.
Substrate behaviour the code already depends on
- Actor template images must be digest-pinned (
@sha256:…); a tag-only
spec.profile.image fails at ActorTemplate creation with reason TemplateCreationFailed (docs/manifests.md). make images writes bin/ax-server.digest / bin/ax-runner.digest for this reason.
- There is no server-wide default task image. A set
spec.profilemust name
image or ateTemplate.name (validateProfile in internal/apivalidation/common.go), a Workspace with setup steps must set one, and a Task that omits it inherits its Workspace's (EffectiveProfile); CreateTask in internal/server/server.go rejects a Task with neither. ResolveProfileTemplate errors on an empty image. E2E Task and Workspace manifests get the image from {{taskImage}} ($AX_E2E_TASK_IMAGE, written to .tmp/e2e.env by hack/setup-e2e-kind.sh).
- After
ResumeActorthe actor may be on a different worker, so AX never
stores a worker IP: the reconciler uses the IP ResumeActor returns only for its readiness probe (internal/controller/reconciler.go), and ax ssh connects through atenet-router.
- Egress: an actor without an egress policy has no outbound access. For
every task, bindGateway (internal/controller/reconciler.go) asks Substrate whether the actor has a policy. If it has none, bindGateway creates its one policy (named default): the runner rule (substrate.RunnerEgressRule), then the rules of the Gateway the task binds, if any. An actor that has a policy keeps it, so the rules are copied once and never updated; Substrate deletes the policy with the actor. bindGateway runs after the actor is suspended (CreateTask, SuspendTask) and before it is resumed (ResumeTask). Substrate validates the rules when they are copied. bindGateway fails with reason GatewayBindingFailed when Substrate refuses them, or when the actor has no policy and the Gateway cannot be loaded. SuspendTask and ResumeTask do not fail on the Gateway themselves (fetchGateway in internal/server/server.go), so a suspend always reaches SuspendActor.
- Substrate's egress gateway reuses an Actor JWT by actor name and audiences
(agent-substrate/substrate#2208), and ax-server's runner endpoint refuses a JWT of another actor UID (internal/runnerhttp/server.go). So bindGateway adds an extra audience, unique to the policy, to the runner rule: a task recreated under the same name then gets a JWT for its new actor. Its first calls can still fail for about 10 seconds, until the gateway reads the new policy.
status.actorUidis read live bysetActorStatusin
internal/server/server.go and cleared by storedStatus before writes. It changes when Substrate recreates the actor and is empty when the actor is missing or cannot be read. ax describe task shows it as Actor UID.
- Guest services (process + fs) are mounted by
internal/metadata/server.go
only when task.Spec.Debug is true; ax ssh (cmd/ax/main.go) refuses to connect otherwise.
- Trust bundle:
BuildActorTemplatemountsegress-mitm.ate.devand
system-roots.ate.dev at v1alpha1.EgressTrustBundlePath for every task, and the runner points the CA variables at it (runner/env.go). A Substrate install that lacks either bundle blocks actor startup.
- Credential injection replaces a header only on requests that carry it. The
runner supplies a placeholder only for git: GIT_ASKPASS is /ax/git-askpass, a link to the running program, and package runner answers from init when a program that imports it is started under that name (runner/env.go). It sets no API key variable, because tools choose a provider by which key is set: a task sets its own placeholders in spec.env. A Gateway's spec is never passed into the sandbox.
- Readiness: Substrate and the reconciler probe the runner on port 80
(/healthz, /readyz?check=workspace, served by internal/metadata); /readyz is 503 until all workspaces are materialized (docs/sandbox.md).
- Workspace snapshot tags:
SuspendActordefaults to asynchronous snapshot
upload, whereas CreateTag requires the source actor's snapshot to be recorded, so WorkspaceManager suspends setup actors with SuspendActorAndWaitForSnapshot (DURABILITY_MODE_SYNCHRONOUS, internal/substrate/client.go). Because CreateTag spans two transactions around an object copy, an interrupted call can leave a tag with status.snapshot unset or SNAPSHOT_STORAGE_STATE_PENDING; isTagCompleted (internal/controller/workspace.go) requires SNAPSHOT_STORAGE_STATE_COMPLETED (populated by internal/substrate/substratetest/server.go in tests).
- Snapshot storage: Actor templates created by AX leave
SnapshotConfig.StorageLocation
empty by default so Substrate uses its cluster-wide --snapshot-location. Specifying a custom snapshot bucket (snapshotsBucket in BuildActorTemplate) can only be used when the Substrate cluster has the PerTemplateSnapshotLocation preview feature gate enabled; otherwise Substrate rejects template creation with actor_template.snapshot_config.storage_location: Forbidden. Since v1.0.0 each path segment may contain only [A-Za-z0-9_-] and no segment may be atespaces; AX passes the value through unvalidated, so Substrate is the one that rejects it.
5. E2E and the Substrate version pin
This is the start of the file. Read the full SKILL.md on GitHub.
Security
- Security scanPassed, 100 of 100Pattern checks on the skill's files. Certified 15 Nov 2026.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingskills/substrate/SKILL.md
Popularity
- 7-day trend score
- 0
Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/google/ax/substrate)