watcher-hermes/hermes-skills/ai-scientist-sandbox-review

Claude Code and agent skill by watcher-hermes/hermes-skills

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Two-gate review checklist for research-loop agent outputs before anything leaves the sandbox.

Install

npx vskill@latest install watcher-hermes/hermes-skills/ai-scientist-sandbox-review

SKILL.md

The same SKILL.md was published earlier as watcher-hermes/hermes-skills/ai-scientist-sandbox-review.

Given an autonomous research output (hypothesis, code, experiments, figures, paper draft) produced by an AI-Scientist-v2-style loop, produce a two-gate review: sandbox audit (does anything leave?) plus research audit (is the work sound?).

The two gates map directly onto the audits below: Sandbox gate = item 1; Research gate = items 2 (Experiment audit) + 3 (Polish audit). Items 4–5 govern what happens after both gates pass.

Produce:

  1. Sandbox gate. Before any artifact leaves the sandbox:
  • List every network call the loop made and its target. Flag any that were not pre-approved.
  • Inventory every file the loop wrote outside its working directory.
  • Confirm Docker / seccomp / gVisor containment held for the full run.
  • Confirm no subprocesses escaped the sandbox's supervision.

If any check fails, block export; raise to a human.

  1. Experiment audit. Read the experiment code, not the paper:
  • Verify every claimed experiment actually ran and its reported numbers are reproducible.
  • Check that failed experiments were reported as failures, not re-framed as negative results after-the-fact.
  • Check that the "novelty" label on the idea holds up against a literature search by a human domain expert.
  1. Polish audit. Read the figures:
  • Ensure every figure's data came from a logged experiment run, not from polish-stage rewriting.
  • Confirm axes, scales, and annotations match the underlying data.
  • Flag any figure whose caption claims more than the data supports.
  1. Disclosure plan. If the artifact is intended for external distribution:
  • Disclose that the artifact is agent-authored.
  • Disclose the tools used (model family, loop version).
  • Disclose the human reviewer who checked it and what they checked.
  1. Negative-release decision. If the artifact fails any audit step, the default is do not release. Overriding this default requires a named human owner.

Hard rejects:

  • Any submission that skips either gate.
  • Any artifact where the loop's execution logs are missing or incomplete.
  • Any figure that cannot be traced to a specific experiment run.
  • Any novelty claim that a domain expert has not verified.

Refusal rules:

  • If the run lacks Docker or equivalent isolation, refuse and require re-run in an isolated sandbox.
  • If the user cannot produce execution logs for the experiment stage, refuse — the paper is unreviewable.
  • If the proposed distribution channel is a peer-reviewed venue and the user proposes not to disclose agent authorship, refuse and require disclosure.

Output format:

Return a two-gate report:

  • Sandbox gate verdict (PASS / BLOCK, with rationale)
  • Research gate verdict (covers Experiment audit (2) and Polish audit (3)) (PASS / BLOCK / REQUIRES_EXPERT, with per-check notes)
  • Disclosure plan (venue, text, human reviewer name)
  • Release decision (release / hold / reject)
  • Next action (who does what by when)

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

7-day trend score
5

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/watcher-hermes/hermes-skills/ai-scientist-sandbox-review/badge)](https://verified-skill.com/skills/watcher-hermes/hermes-skills/ai-scientist-sandbox-review)

Something wrong with this skill or its page? Tell us.