caidish/klayoutclaw/e2e-judge

Claude Code and agent skill by caidish/klayoutclaw

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Run agentic E2E tests for KlayoutClaw. Gives Claude Code natural-language tasks, lets it autonomously choose MCP tools, independently verifies layout state via direct MCP calls, and judges outcomes with gpt-5-mini. Use when the user wants to run agentic tests, validate KlayoutClaw, or benchmark agent performance.

Install

npx vskill@latest install caidish/klayoutclaw/e2e-judge

SKILL.md

Agentic E2E Test for KlayoutClaw

End-to-end agentic test suite. Each test gives Claude Code a natural-language task (not a hardcoded command), lets the agent autonomously decide which MCP tools to call, then independently verifies the layout state and judges the outcome with an LLM.

Prerequisites

  • KLayout running with KlayoutClaw plugin (MCP on 127.0.0.1:8765)
  • claude CLI on PATH (Claude Code)
  • Network access to http://api.physcai.com (LLM judge)

Architecture

Per test:
1. [Optional] Reset layout via direct MCP call
2. run_agent(task) → claude --print --mcp-config → AgentResult(transcript)
3. run_verification(fn, client) → VerificationResult(checks, passed)
4. Compose context = transcript + verification
5. judge(context, assertion) → JudgeVerdict(passed, reasoning, confidence)
FileRole
harness.pyInvokes claude --print --mcp-config with NL prompts
verifier.pyIndependent MCP state checks (cells, shapes, layers, files)
verifier_phase5.pyPhase 5 verification functions (19 functions for pipeline/eval tests)
judge.pyLLM judge — gpt-5-mini via api.physcai.com
conftest.pyAgenticTestCase/Result dataclasses, formatting
run_tests.py15 agentic tests across 6 groups (Phase 1-4)
run_tests_phase5.py26 agentic tests across 9 groups (Phase 5: autonomous pipeline)

Scripts

bash
# Phase 1-4 tests (15 tests)
python scripts/run_tests.py                  # run all 15 tests
python scripts/run_tests.py --test T2.1      # single test
python scripts/run_tests.py --group recon    # one group
python scripts/run_tests.py --json           # JSON output
python scripts/run_tests.py --list           # list tests
 
# Phase 5 tests (26 tests)
python scripts/run_tests_phase5.py                        # run all 26 tests
python scripts/run_tests_phase5.py --test T0.1            # single test
python scripts/run_tests_phase5.py --group preflight      # one group
python scripts/run_tests_phase5.py --json                 # JSON output
python scripts/run_tests_phase5.py --list                 # list tests
python scripts/run_tests_phase5.py --nightly              # include optional nightly tests (T10.2)

Test Groups (41 tests total)

Phase 1-4 (15 tests via run_tests.py)
GroupTestsWhat it tests
reconT0.1-T0.2Agent explores tools, inspects layout state
layoutT1.1-T1.3Agent creates layouts and cells from NL specs
geometryT2.1-T2.3Agent draws rectangles, polygons, paths from descriptions
composeT3.1-T3.3Agent performs multi-step: create cell + instance, multi-layer, query + modify
exportT4.1-T4.2Agent saves GDS and takes screenshots
deviceT5.1-T5.2Agent designs complete Hall bar from high-level spec
Phase 5 (26 tests via run_tests_phase5.py)
GroupTestsWhat it tests
preflightT0.1-T0.3Tool/skill discovery triad, skill doc reading, validate_pixel_size
inspectionT1.1-T1.3Per-layer shape audit, pixel gate valid/invalid handling
evaluateT2.1-T2.3evaluate_design DRC mode, score mode, graceful error on missing ref
hallbarT3.1-T3.3Hall bar full flow on synthetic flake, topgate-off variant, deliverables package
pipelineT4.1-T4.3Pipeline step ordering (QUERY+VALIDATE only), auto_route layer strings, array layer map
discoveryT6.1-T6.3Skill inventory, hallbar Step 0 checklist, E2E gate plan extraction
layout_deepT7.1-T7.3Per-layer metrics, recursive cell hierarchy, text+polygon semantics
evaluate_advT9.1-T9.3DRC advanced analysis, score mode advanced, dual-mode consistency report
full_pipelineT10.1(-T10.2)Hall bar on precomputed contours, full E2E pipeline (T10.2 nightly only)

Key difference from coding tests

Coding test: python add_rect.py CELL 1 0 -50 -25 50 25 → check stdout Agentic test: "Draw a 100x50um rectangle on layer 1/0" → agent decides how → verify layout state

The agent is free to use execute_script, skill scripts, or any combination. We only verify the outcome, not the method.

Configuration

VariableDefaultDescription
E2E_AGENT_MODELsonnetClaude model for agent invocations
LLM_JUDGE_API_BASEhttp://api.physcai.comJudge API endpoint
LLM_JUDGE_API_KEY(built-in)Judge API key
LLM_JUDGE_MODELgpt-5-miniJudge model
KLAYOUT_MCP_URLhttp://127.0.0.1:8765/mcpMCP server URL

Security

  • Security scanPassed, 90 of 100Pattern checks on the skill's files.
  • Model reviewPassed

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

Repository stars
40
Repository forks
7
7-day trend score
0

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/caidish/klayoutclaw/e2e-judge/badge)](https://verified-skill.com/skills/caidish/klayoutclaw/e2e-judge)

Something wrong with this skill or its page? Tell us.