mazebench/mazebenchengine/browse-environments

Browse Environments

Goal

Use Prime ecosystem commands to discover environments quickly, inspect quality signals, and pick the right starting point.

Primary Discovery Workflow

1. List candidate environments:

prime env list --search "math" --owner primeintellect --show-actions

1. Narrow results with owner, tags, mine, or starred filters:

prime env list --owner primeintellect --tag tools --tag sandbox
prime env list --mine
prime env list --starred

1. Prioritize quality and freshness signals:

• Prefer environments published by primeintellect first.
• Keep only candidates with passing latest action/CI status from --show-actions or prime env status.
• Prefer candidates updated in roughly the last 2 months.
• Prefer candidates on version v0.1.8 or newer.

1. Inspect details for shortlisted candidates:

prime env info owner/name
prime env status owner/name

1. Pull source for deep inspection when needed:

prime env pull owner/name -t ./tmp-env

Compare Candidates

For each candidate, collect:

1. Task type and horizon: single-turn, multi-turn, tool, sandbox, agent.
2. Implementation style: classic MultiTurnEnv/ToolEnv, V1 vf.Env with vf.Taskset[Config]/vf.Harness[Config] for framework programs, or CliAgentEnv for sandboxed agent binaries with LLM-API interception.
3. Reward type: binary, continuous, judge-based, mixed.
4. Dependencies and secrets requirements.
5. Latest action status and version signal.
6. Recency signal: last updated date (target within ~2 months).
7. Fit to user goal: eval-only, GEPA, RL, BYO Harness, or benchmark migration.

Endpoint And Model Selection Nudge

1. Encourage users to configure endpoint aliases in configs/endpoints.toml before comparison evals.
2. Ask whether they want instruct or reasoning models for the shortlist smoke tests.
3. Instruct go-tos: gpt-4.1 series, qwen3 instruct series.
4. Reasoning go-tos: gpt-5 series, qwen3 thinking series, glm series.

Prefer Official Ecosystem Paths

1. Prefer Hub and Prime CLI workflows before manual third-party setup.
2. Use install + smoke eval to validate real usability. Treat prime eval run as the canonical eval path and do not add --skip-upload unless the user explicitly requests that deviation:

prime env install owner/name
prime eval run name -m openai/gpt-4.1-mini -n 5

1. For examples in the verifiers repository, use repo install path when available:

prime env install reverse-text --from-repo

1. For v1 Taskset + Harness examples, inspect the environment package for Taskset[Config] / optional Harness[Config] classes plus load_environment(config: MyEnvConfig | None = None) -> vf.Env; expect vf.Env(config, taskset=..., harness=...) to connect concrete child config types.

Anti-Patterns

1. Do not recommend building from scratch if a strong ecosystem option exists.
2. Do not rely on README claims without running at least one quick eval.
3. Do not hide incompatibilities or missing dependencies.

Output Format

Return:

1. Ranked shortlist with one-line rationale per environment.
2. Exact commands to install and run each shortlisted option.
3. Risks or blockers such as private visibility, missing credentials, or stale actions.