dosco/graphjin/graphjin-env

GraphJin Agent Environment

Use this skill when a user wants to train or measure an agent against a GraphJin environment: running the container, driving graded episodes, collecting trajectories, or interpreting a reward.

For creating and running evaluation suites against a project, use the graphjin-eval skill instead. This one is about the environment as a training target.

This skill deliberately states no task counts, no flag defaults and no measured figures. Those live in /health and at <https://graphjin.com/environment/>. Read them from the running server rather than from here — a skill that carries no facts cannot carry stale ones.

Rules

• **Read /health before anything else, and record it with any number you

report.** A reward is only comparable against another from the same world under the same contract. The fields that decide it: reward_version, reward_profile, dataset.catalog_hash, dataset.data_anchor, capabilities.suite_fingerprint, and build.version.

• Check `capabilities.catalog_match` before a long run. If it is false,

the suite's oracles were verified against a different schema and every episode is being graded against the wrong answers. If it is absent, there was nothing to compare.

• Never compare an external-mode reward with a hosted one. An external

agent's token use never reaches the server, so the efficiency term is unmeasured rather than zero.

• Never present a small difference as a result. The same binary run twice

against the same suite flips a meaningful number of tasks. Use training/measure.py, which prints a confidence interval and the resolution floor, and quote both.

• A `GJ_ENV_` variable the server does not read is a startup error, not a

default. If the server refuses to start, read the error — it names the variable.

• Without a temperature, a sampling group returns n identical answers. The

stack pins sampling to zero. If a group comes back with identical rewards, check this before concluding anything about the model.

• Never hand-edit eval/suite.yml, eval/suite.split.json, world-pack.json,

or any file under .graphjin-evals/. Each carries a fingerprint something downstream compares against.

• Never spend provider tokens without the user's approval. Commands that call a

model require --yes and print the call count first; surface that preview.

Workflow

1. Start it and read what it is.

``sh docker run -d -p 8090:8090 --tmpfs /tmp:size=1g dosco/graphjin:env-latest curl -s localhost:8090/health ``

/tmp must be writable — each world provisions its own database there. Confirm status, capabilities.catalog_match, and capabilities.drive_modes before going further.

1. Pick a drive mode from `capabilities.drive_modes`, not from assumption:

Endpoints that were not enabled do not exist; a 404 means the flag was not given.

1. Keep held-out work held out. Serve --split auto:0.8 --side train for

collection and --side eval for measurement. graphjin eval export refuses to build a training corpus from eval-side episodes and says how many it found; --allow-eval-side overrides it, and using that override silently is how a number stops meaning generalization.

1. Collect, then convert.

``sh graphjin eval sample --repeats 8 --temperature 0.8 --split <split> --side train --yes graphjin eval export <run-id> --split <split> --side train --stage executor --out run.jsonl ``

--stage executor matters: an agent run is three model calls with different jobs, and mixing them into one corpus teaches none of them.

1. Measure on the held-out side between checkpoints, never on the training

side, and quote the interval rather than the point estimate.

Diagnosis

• Every episode scores zero — usually no model configured, or a base model

that skips discovery. Check GJ_AGENT_*; then run one episode with include_response and read what it actually did.

• The server refuses to start — read the message. A refused suite, a

catalog mismatch, an unreadable GJ_ENV_ variable and an unwritable work directory each say which one they are.

• Rewards identical across a sampling group — no temperature, or a provider

that manages sampling itself.

• `410` from a step or external route — the episode ended or timed out and

its world was reclaimed. Start a new one; do not retry the old id.

• A number moved a little between checkpoints — that is the instrument, not

the model. See the resolution floor.

Reference

<https://graphjin.com/environment/> — quickstart, drive modes, training workflows, reward and comparability, CLI, HTTP API, and file formats.