tommy-yw/runbookhermes/hermes-atropos-environments
Claude Code and agent skill by tommy-yw/runbookhermes
- community
- ext:[{"type":"config","description":"Configuration overrides"}]
Checking that the SKILL.md file is still in the repository.
Build, test, and debug Hermes Agent RL environments for Atropos training. Covers the HermesAgentBaseEnv interface, reward functions, agent loop integration, evaluation with tools, wandb logging, and the three CLI modes (serve/process/evaluate). Use when creating, reviewing, or fixing RL environments in the hermes-agent repo.
Install
npx vskill@latest install tommy-yw/runbookhermes/hermes-atropos-environmentsSemi-extensibleMentions customization, but needs skill-specific knowledge. Read the extensible skills standard
SKILL.md
The same SKILL.md was published earlier as tommy-yw/runbookhermes/hermes-atropos-environments.
Hermes Agent Atropos Environments
Guide for building RL environments in the hermes-agent repo that integrate with the Atropos training framework.
Architecture Overview
Hermes environments are special because they run a multi-turn agent loop with tool calling — not just single-turn completions. The base env handles the loop; you implement the task and scoring.
File Locations
| File | Purpose | |
|---|---|---|
environments/hermes_base_env.py | Base class with agent loop + tool resolution | |
environments/agent_loop.py | HermesAgentLoop + AgentResult dataclass | |
environments/tool_context.py | ToolContext for reward verification | |
environments/tool_call_parsers.py | Phase 2 tool call parsers (hermes, mistral, etc.) | |
environments/your_env.py | Your environment implementation |
Inference Setup — Ask the User First
IMPORTANT: Before running any test, evaluation, or data generation command, always ask the user how they want to handle inference. Do NOT assume OpenRouter or any specific endpoint. Present these options:
- OpenRouter — Ask which model they want to use (e.g.,
anthropic/claude-sonnet-4.5,google/gemini-2.5-pro,meta-llama/llama-3.3-70b-instruct, etc.). RequiresOPENROUTER_API_KEYin environment. - Self-hosted VLLM endpoint — Ask for their base URL (e.g.,
http://localhost:8000/v1) and model name. Set--openai.server_type vllm. - Other OpenAI-compatible API — Ask for the base URL, model name, and any required API key. Set
--openai.server_type openaiand--openai.health_check false. - Local Atropos training server — For
servemode with a live training loop. Defaulthttp://localhost:8000/v1.
Once the user tells you their setup, use those values in all CLI commands for that session. Example prompts:
"Before I run this, how would you like to handle inference?1. OpenRouter (I'll need your preferred model, e.g. claude-sonnet-4.5)2. A self-hosted VLLM endpoint (give me the URL and model name)3. Another OpenAI-compatible API (give me the URL, model, and any auth details)4. Local Atropos training server (serve mode)"
Key flags by provider:
| Provider | --openai.server_type | --openai.health_check | --openai.api_key | |
|---|---|---|---|---|
| OpenRouter | openai | false | $OPENROUTER_API_KEY | |
| VLLM (self-hosted) | vllm | (default) | (not needed) | |
| Other OpenAI-compatible | openai | false | As needed | |
| Local Atropos | (default) | (default) | (not needed) |
Required Methods
1. setup() — Load dataset and initialize state
2. get_next_item() — Return next training item
3. format_prompt(item) — Convert item to user message
4. compute_reward(item, result, ctx) — Score the rollout
CRITICAL: result is an AgentResult, NOT a dict. It has these attributes:
result.messages— List of message dicts (OpenAI format)result.turns_used— Number of LLM calls maderesult.finished_naturally— True if model stopped voluntarilyresult.tool_errors— List of ToolError objects
AgentResult does NOT have: final_response, tool_calls, tools_used. You must extract these from result.messages:
ctx (ToolContext) gives you terminal/file access to the agent's sandbox for verification:
5. evaluate() — Periodic evaluation with full agent loop
MUST use the full agent loop with tools, not single-turn chat_completion. The whole point of hermes-agent environments is agentic evaluation:
6. wandb_log() — Custom metrics logging
Always call super().wandb_log() at the end:
Pitfall: compute_reward appends to metric buffers. During eval, this pollutes training metrics. Roll back buffer entries added during eval.
Config Class
Always create a custom config subclass with Pydantic Field descriptors. Key inherited fields you can tune: enabled_toolsets, max_agent_turns, agent_temperature, system_prompt, terminal_backend, group_size, steps_per_eval, total_steps.
config_init() — Default Configuration
Classmethod returning (YourEnvConfig, [APIServerConfig(...)]). Set server_type to "openai" for OpenRouter/external APIs. Load API key from environment variable.
Three CLI Modes
Config priority: CLI args > YAML file > config_init() defaults.
Common Pitfalls
- AgentResult has .messages, not .final_response — Extract the final response by iterating reversed(result.messages) looking for the last assistant message with content.
- evaluate() must use HermesAgentLoop, not chat_completion — Single-turn chat_completion has no tools. The whole point of hermes-agent benchmarks is agentic evaluation with tool use.
- Don't call _llm_judge twice — If compute_reward already calls it, extract the score from the buffer instead of calling judge separately in evaluate().
- Eval pollutes training buffers — compute_reward appends to metric buffers. During eval, roll back buffer entries to keep training metrics clean.
- Always set health_check=false for OpenRouter — OpenRouter has no /health endpoint.
- Set data_dir_to_save_evals in evaluate mode — Without it, results aren't saved.
- default_toolsets class variable vs enabled_toolsets config — The class variable is a hint; the config field is what actually controls tool resolution.
- Tool call parsing in messages — Tool calls are dicts with
{"function": {"name": ..., "arguments": ...}}. Always checkisinstance(tc, dict).
- ToolContext.cleanup() — Always call in a finally block to release sandbox resources.
- server_type must be "openai" for external APIs — Without it, Atropos assumes a local VLLM server.
- Always ask the user for their inference setup — Never hardcode or assume a specific provider/model. See the "Inference Setup" section above.
Reward Function Patterns
LLM Judge (for open-ended tasks)
Use self.server.chat_completion() with a scoring prompt. Parse JSON response for score float. Always include a heuristic fallback (keyword overlap) for when the judge call fails.
Binary Verification (for code/terminal tasks)
Use ctx.terminal("pytest test.py -q") to run tests in the agent's sandbox. Return 1.0 for pass, 0.0 for fail.
Multi-Signal (combine multiple indicators)
Weight correctness (0.6) + tool usage (0.2) + efficiency (0.2) + optional bonuses. Clamp to [0, 1].
Testing Your Environment
- Import test:
python -c "from environments.my_env import MyEnv; print('OK')" - Ask the user for inference setup (see "Inference Setup" section above)
- Process mode (1 item): Verify JSONL output has valid tokens, masks, scores
- Evaluate mode: Verify full agent loop runs with tools, metrics logged correctly
- Check reward range: Scores should be in [0, 1], not all identical
This is the start of the file. Read the full SKILL.md on GitHub.
Security
- Security scanPassed, 95 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingoptional-skills/mlops/hermes-atropos-environments/SKILL.md
Popularity
- Repository stars
- 547
- Repository forks
- 41
- 7-day trend score
- 32
Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.
Extension points
configConfiguration overrides
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/tommy-yw/runbookhermes/hermes-atropos-environments)