deepx-ai/dx-all-suite/dx-internal-model-eval

Claude Code and agent skill by deepx-ai/dx-all-suite

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Internal skill — evaluate and compare coding-agent / model performance via the e2e-test autopilot harness. Use when comparing coding agents (claude-code vs copilot vs cursor vs opencode vs codex), comparing models (e.g. a new frontier model vs the existing one, "opus4.8 vs fable5"), or comparing thinking / reasoning-effort levels (TH vs NTH). Triggers (EN): "model eval", "coding agent comparison", "frontier model", "thinking mode comparison". Triggers (KO): "신규/프론티어 모델 성능 비교", "thinking 모드 비교", "reasoning effort 비교", "e2e 비교평가". <!-- KOREAN-OK: KO trigger phrases enable Korean-prompt auto-detection --> Explicit: /dx-internal-model-eval.

Install

npx vskill@latest install deepx-ai/dx-all-suite/dx-internal-model-eval

SKILL.md

Skill: dx-internal-model-eval

Internal-business skill. Drives the existing tools (.deepx/e2e/e2e_runner.py,
.deepx/e2e/agent_analyzer/) — it does not reimplement them. Full detail in
reference.md (reference-KO.md for Korean). Ledger contract in registry-schema.md.

When to use — three eval purposes

PurposeMechanism
Coding-agent comparisone2e_runner --tools <all> → multi-tool analyze
Model comparison (new frontier vs existing)e2e_runner --<tool>-model OLD vs NEW → 2 run_ids → build_comparison
Thinking / reasoning-effort comparisone2e_runner --thinking vs non → build_comparison

Procedure (one cycle)

  1. Confirm: purpose, tool scope (--tools), OLD/NEW model ids (hyphen for claude CLI,

dot for copilot), round count, thinking on/off.

  1. Run each model/condition via the resilient controller (auto-recovers from usage limits):

python3 .deepx/e2e/e2e_resilient_run.py --tool claude-code --model <id> --rounds N [--thinking] (wraps e2e_runner with redo-env-failures + wait-for-reset + --resume; see reference.md "Usage-limit resilience"). Record each run_id ↔ model.

  1. Analyze each run_id, writing into the durable archive (never the gitignored worktree):

python3 analyze.py --run-id <RID> --output-dir "$DX_MODEL_EVAL_ARCHIVE/<label>/" where DX_MODEL_EVAL_ARCHIVE defaults to $HOME/shared/coding_agent_diff_report.

  1. Compare: build_comparison.py between the two report dirs (mind the Thinking/Non-Thinking

label caveat — see reference.md), or multi-run analyze.

  1. Bundle raw results for durability: `python3 .deepx/e2e/bundle_raw_results.py

--results-dir <results/<run_id>> --out "$DX_MODEL_EVAL_ARCHIVE/<label>/raw/" (excludes .dxnn/venv/*.onnx`; see reference.md for the symlink/scattered-dir rationale).

  1. Append one ledger row per cycle per registry-schema.md.

Hard rules

  • Reports + bundles ALWAYS go to $DX_MODEL_EVAL_ARCHIVE (worktree-local

dx-agent-dev/e2e-tests/ is gitignored and lost on worktree cleanup).

  • Attribute models by run_id, NOT the CSV model column (it can fall back to config

default — see reference.md §6.2).

  • Do not change analyze.py's built-in default output dir; override per-invocation.

See also

  • reference.md / reference-KO.md — full command reference, caveats, past-report index.
  • registry-schema.md — ledger columns + append procedure.

Security

  • Security scanPassed, 90 of 100Pattern checks on the skill's files.
  • Model reviewPassed

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

Repository stars
44
Repository forks
20
7-day trend score
19

Stars and forks are for the whole GitHub repository, not this skill alone. Refreshed 6 Oct 2026, a month ago.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/deepx-ai/dx-all-suite/dx-internal-model-eval/badge)](https://verified-skill.com/skills/deepx-ai/dx-all-suite/dx-internal-model-eval)

Something wrong with this skill or its page? Tell us.