google/ax/substrate

Claude Code and agent skill by google/ax

Trust tier 4
  • vendor
Source check pending

Checking that the SKILL.md file is still in the repository.

How AX depends on Agent Substrate (modules, address contract, mocks, version bumps, E2E pin). Load when touching internal/substrate, bumping Substrate, or triaging make test-e2e.

Install

npx vskill@latest install google/ax/substrate

SKILL.md

The same SKILL.md was published earlier as google/ax/substrate.

Agent Substrate in AX

AX schedules every Task as a sandboxed actor on Agent Substrate. AX owns the declarative layer (Task/Workspace/Model, reconciler, CLI); Substrate owns isolation, checkpoint/restore, worker placement and in-cluster routing. Facts below were verified against main on 2026-11-06 — re-check go.mod and hack/setup-e2e-kind.sh if they look stale.

1. Module map

ModulePinnedProvidesImported by
go.ate.dev/substratev1.0.0Control API protos api/ateapi/v1 (atespaces, actor templates, actors, tags, snapshot fidelity, suspend/resume, egress rules) and pkg/postgressetupinternal/substrate, internal/substrate/substratetest, internal/install/substrate, internal/server, internal/controller, internal/apivalidation, test/e2e, pkg/apis/v1alpha1 (its .proto files import ateapi.proto)
github.com/agent-substrate/envv0.0.11-0.20260912052224-4468a200b170 (pseudo-version)Guest services served inside the sandbox (process exec + file system gRPC) and proto/ateenv/v1alphainternal/guest, internal/metadata

There is no replace directive for either module. Local hacking against a Substrate checkout goes through SUBSTRATE_DIR in E2E (section 5).

2. The Control API wrapper: internal/substrate.Client

All Control API traffic goes through this package. It encapsulates:

  • Endpoint defaults — api.ate-system.svc.cluster.local:443; authority

and TLS ServerName default to api.ate-system.svc when the target contains that string.

  • Auth — bearer token from SUBSTRATE_TOKEN_FILE (in-cluster: projected SA

token with audience api.ate-system.svc, see deploy/ax-server.yaml), CA from SUBSTRATE_CA_FILE.

  • Retry — IsTransientError (Unavailable, DeadlineExceeded, Aborted) + callWithRetry (retryOnCodes).
  • Helpers the reconcilers rely on — EnsureAtespace, EnsureActorTemplate,

EnsureActorTemplateWithImage, BuildActorTemplate, EnsureActor, EnsureActorFromTag, GetActorEgressPolicy, EnsureActorEgressPolicy, CreateTag, GetTag, ListTags, DeleteTag, SuspendActor, SuspendActorAndWaitForSnapshot, ResumeActor, DeleteActor, WorkerIP, ResourceLimits.

Where a new Control API call lives today
  • internal/substrate/client.go is the only non-test file that calls

ateapipb.NewControlClient; every existing RPC is a method on *Client, wrapped in callWithRetry (codes.Unavailable plus optional codes such as codes.Aborted).

  • Each method has a table-driven case in client_test.go (mockControlClient).
  • Callers in internal/controller exercise it through mockControlServer

(reconciler_test.go) or internal/substrate/substratetest (workspace_test.go); internal/server and cmd/ax tests also use internal/substrate/substratetest (section 4).

3. Address contract

ValueOccurrences
api.ate-system.svc.cluster.local:443 + authority api.ate-system.svccmd/ax-server/main.go (--substrate-endpoint / --substrate-authority flag defaults), internal/substrate/client.go (3×), deploy/ax-server.yaml, test/e2e/manifests/ax-server.yaml, test/e2e/harness_test.go
atenet-router.ate-system.svc[.cluster.local]internal/controller/reconciler.go (defaultRouterServerName), internal/tunnel/router_tls.go (routerServerName), cmd/ax/main.go (ssh port-forward path), internal/guest/client.go (doc comment), deploy/ax-server.yaml, test/e2e/manifests/ax-server.yaml, docs/networking.md
namespace ate-systemall of the above + test/e2e/harness_test.go (ateSystemNamespace), test/e2e/manifests/credential-policy.yaml, internal/install/axdeploy/authz.go, internal/install/cluster/kind.go, internal/install/substrate/bootstrap.go, internal/install/substrate/pki.go, hack/sync-substrate-manifests.sh (and generated internal/install/substrate/manifests/*.yaml)
pod labels ate.dev/worker-pool (worker pods) and app: atenet-egress (egress gateway)the ax-server NetworkPolicy in deploy/ax-server.yaml and test/e2e/manifests/ax-server.yaml, which blocks both from :8080/:9090 and allows only app: atenet-egress (in ate-system) to reach :8081 (ax-server-runner)
opentelemetry-collector.otel-system.svc:4317 (the collector Substrate's kind overlay deploys, manifests/ate-install/kind/ate-otel-config.yaml in Substrate)test/e2e/manifests/ax-server.yaml (ax-otel-config ConfigMap). An unreachable address does not fail E2E; ax-server only logs OTLP export errors
kind-registry:5000, the registry container that serves the images of a loopback registry (localhost:5001/...) to an ax local clusterOnly atelet's --localhost-registry-replacement in internal/install/substrate/manifests/05-ate-system.yaml (generated; actor images). LocalRegistryEndpoint (internal/install/substrate/bootstrap.go) reads it from there, and cmd/ax/install_cmd.go passes it as cluster.UpOptions.LocalRegistry to the Kind provider (internal/install/cluster/registry.go), which writes the containerd mirror for pod images such as ax-server. No Go file names the registry; docs/local_installer.md does

Why it matters: tasks never get a Service/Ingress. Every request to a task is proxied by the atenet router, which picks the target from the ate-target-actor header and resumes a suspended actor on demand (docs/networking.md). ax ssh port-forwards to svc/atenet-router on 443 (mTLS) and falls back to 80.

4. Testing against Substrate

Unit tests: in-process mock, no cluster
go
type mockControlServer struct {
ateapipb.UnimplementedControlServer
// record calls / inject failures here
}
grpcServer := grpc.NewServer()
ateapipb.RegisterControlServer(grpcServer, mockSrv)
// serve on a listener, point substrate.NewClient at it

Canonical examples: internal/controller/reconciler_test.go (reconciler state transitions; the largest test file in the repo). internal/server/server_test.go and cmd/ax/main_test.go run a real TaskReconciler against internal/substrate/substratetest, an in-memory fake of the same API (atespaces, actor templates, actors, and tags). internal/substrate/client_test.go uses a client-side mockControlClient instead. No mocking library is in go.mod.

Substrate behaviour the code already depends on
  • Actor template images must be digest-pinned (@sha256:…); a tag-only

spec.profile.image fails at ActorTemplate creation with reason TemplateCreationFailed (docs/manifests.md). make images writes bin/ax-server.digest / bin/ax-runner.digest for this reason.

  • There is no server-wide default task image. A set spec.profile must name

image or ateTemplate.name (validateProfile in internal/apivalidation/common.go), a Workspace with setup steps must set one, and a Task that omits it inherits its Workspace's (EffectiveProfile); CreateTask in internal/server/server.go rejects a Task with neither. ResolveProfileTemplate errors on an empty image. E2E Task and Workspace manifests get the image from {{taskImage}} ($AX_E2E_TASK_IMAGE, written to .tmp/e2e.env by hack/setup-e2e-kind.sh).

  • After ResumeActor the actor may be on a different worker, so AX never

stores a worker IP: the reconciler uses the IP ResumeActor returns only for its readiness probe (internal/controller/reconciler.go), and ax ssh connects through atenet-router.

  • Egress: an actor without an egress policy has no outbound access. For

every task, bindGateway (internal/controller/reconciler.go) asks Substrate whether the actor has a policy. If it has none, bindGateway creates its one policy (named default): the runner rule (substrate.RunnerEgressRule), then the rules of the Gateway the task binds, if any. An actor that has a policy keeps it, so the rules are copied once and never updated; Substrate deletes the policy with the actor. bindGateway runs after the actor is suspended (CreateTask, SuspendTask) and before it is resumed (ResumeTask). Substrate validates the rules when they are copied. bindGateway fails with reason GatewayBindingFailed when Substrate refuses them, or when the actor has no policy and the Gateway cannot be loaded. SuspendTask and ResumeTask do not fail on the Gateway themselves (fetchGateway in internal/server/server.go), so a suspend always reaches SuspendActor.

  • Substrate's egress gateway reuses an Actor JWT by actor name and audiences

(agent-substrate/substrate#2208), and ax-server's runner endpoint refuses a JWT of another actor UID (internal/runnerhttp/server.go). So bindGateway adds an extra audience, unique to the policy, to the runner rule: a task recreated under the same name then gets a JWT for its new actor. Its first calls can still fail for about 10 seconds, until the gateway reads the new policy.

  • status.actorUid is read live by setActorStatus in

internal/server/server.go and cleared by storedStatus before writes. It changes when Substrate recreates the actor and is empty when the actor is missing or cannot be read. ax describe task shows it as Actor UID.

  • Guest services (process + fs) are mounted by internal/metadata/server.go

only when task.Spec.Debug is true; ax ssh (cmd/ax/main.go) refuses to connect otherwise.

  • Trust bundle: BuildActorTemplate mounts egress-mitm.ate.dev and

system-roots.ate.dev at v1alpha1.EgressTrustBundlePath for every task, and the runner points the CA variables at it (runner/env.go). A Substrate install that lacks either bundle blocks actor startup.

  • Credential injection replaces a header only on requests that carry it. The

runner supplies a placeholder only for git: GIT_ASKPASS is /ax/git-askpass, a link to the running program, and package runner answers from init when a program that imports it is started under that name (runner/env.go). It sets no API key variable, because tools choose a provider by which key is set: a task sets its own placeholders in spec.env. A Gateway's spec is never passed into the sandbox.

  • Readiness: Substrate and the reconciler probe the runner on port 80

(/healthz, /readyz?check=workspace, served by internal/metadata); /readyz is 503 until all workspaces are materialized (docs/sandbox.md).

  • Workspace snapshot tags: SuspendActor defaults to asynchronous snapshot

upload, whereas CreateTag requires the source actor's snapshot to be recorded, so WorkspaceManager suspends setup actors with SuspendActorAndWaitForSnapshot (DURABILITY_MODE_SYNCHRONOUS, internal/substrate/client.go). Because CreateTag spans two transactions around an object copy, an interrupted call can leave a tag with status.snapshot unset or SNAPSHOT_STORAGE_STATE_PENDING; isTagCompleted (internal/controller/workspace.go) requires SNAPSHOT_STORAGE_STATE_COMPLETED (populated by internal/substrate/substratetest/server.go in tests).

  • Snapshot storage: Actor templates created by AX leave SnapshotConfig.StorageLocation

empty by default so Substrate uses its cluster-wide --snapshot-location. Specifying a custom snapshot bucket (snapshotsBucket in BuildActorTemplate) can only be used when the Substrate cluster has the PerTemplateSnapshotLocation preview feature gate enabled; otherwise Substrate rejects template creation with actor_template.snapshot_config.storage_location: Forbidden. Since v1.0.0 each path segment may contain only [A-Za-z0-9_-] and no segment may be atespaces; AX passes the value through unvalidated, so Substrate is the one that rejects it.

5. E2E and the Substrate version pin

This is the start of the file. Read the full SKILL.md on GitHub.

Security

  • Security scanPassed, 100 of 100Pattern checks on the skill's files. Certified 15 Nov 2026.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

7-day trend score
0

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill CERTIFIED badge
[![vskill CERTIFIED](https://verified-skill.com/api/v1/skills/google/ax/substrate/badge)](https://verified-skill.com/skills/google/ax/substrate)

Something wrong with this skill or its page? Tell us.