kalash33/kstack/agent-eval
Claude Code and agent skill by kalash33/kstack
Trust tier 3
- community
Source check pending
Checking that the SKILL.md file is still in the repository.
Test and benchmark agent performance. Covers behavioral testing, accuracy measurement, latency profiling, cost tracking, and regression detection. Use when: "test the agent", "benchmark", "is the agent working", "eval", "agent performance", or after building/modifying an agent.
Install
npx vskill@latest install kalash33/kstack/agent-evalSKILL.md
The same SKILL.md was published earlier as kalash33/kstack/agent-eval.
/agent-eval — Agent Evaluator
Measure what matters. Catch regressions before users do.
When to Use
- After building or modifying an agent
- Before deploying to production
- Comparing model/prompt changes
- Tracking quality over time
Evaluation Dimensions
| Dimension | What to measure | |
|---|---|---|
| Accuracy | Does it get the right answer? | |
| Tool use | Does it call the right tools with right args? | |
| Reliability | Does it work consistently (not 1/10 times)? | |
| Latency | How long does it take? | |
| Cost | How many tokens/API calls? | |
| Safety | Does it refuse harmful requests? |
Process
Step 1: Create Test Cases
Step 2: Run Evaluation
Step 3: Score
- Pass/Fail: Binary — did it do the right thing?
- LLM-as-Judge: Use a stronger model to grade output quality
- Human-in-the-loop: Flag uncertain cases for manual review
Step 4: Report
Chaining
- Before:
/agent-build(build first, then eval) - After:
/agent-buildagain (fix failures),/ship(deploy if passing)
Security
- Security scanPassed, 95 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingskills/agent-eval/SKILL.md
Popularity
- 7-day trend score
- 5
Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/kalash33/kstack/agent-eval)