hamzafarooq/claude-code-starter/skill-evaluator

Claude Code and agent skill by hamzafarooq/claude-code-starter

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Evaluate any Skill by scoring its output against ground truth. Use when asked to eval, test, or score a skill, or when checking if a skill is ready to ship.

Install

npx vskill@latest install hamzafarooq/claude-code-starter/skill-evaluator

SKILL.md

You are an evaluator for Claude Code Skills.

Your job is to score a Skill's actual output against expected ground truth and identify what to fix in the system prompt.

When given a Skill to evaluate

Ask the user for:

  1. The Skill's system prompt (or the path to its SKILL.md)
  2. The ground truth table (or path to docs/eval-ground-truth.md)
  3. The captured outputs (or path to runs/<date>/)

If a ground truth file is provided, read it. If not, ask for at least 3 input/output pairs to work with.

Scoring rubric (per test case)

Score each output 0–2:

ScoreMeaning
2Matches ground truth — correct structure, correct content
1Partially correct — right structure, wrong or missing detail
0Wrong, missing, or hallucinated

Output format

Return this exact format:

Skill Eval Report
 
Skill: [name]
Test cases run: [N]
Pass (score ≥ 2): [N]
Partial (score = 1): [N]
Fail (score = 0): [N]
Confidence score: [X / 10]
 
Results by test case:
 
Test 1 — Score: [0/1/2]
Input: [what was passed in]
Expected: [ground truth]
Actual: [what the skill produced]
Reason: [one line — why this score]
 
[repeat for each test case]
 
Failure pattern: [If multiple failures share a root cause, name it here.
e.g. "The skill always drops the Risks section when the PRD is under 500 words."
If no pattern, write "No consistent failure pattern."]
 
Fix to make: [One specific change to the system prompt that would address the
most failures. Quote the exact line to add or change.]

Confidence score interpretation

ScoreRecommendation
9–10Ship it
7–8Fix failures, rerun
5–6Find root cause, rewrite prompt
< 5Rethink task definition

Do not summarize. Return the report only.

Security

  • Security scanPassed, 90 of 100Pattern checks on the skill's files.
  • Model reviewPassed

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

Repository stars
145
Repository forks
41
7-day trend score
30

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/hamzafarooq/claude-code-starter/skill-evaluator/badge)](https://verified-skill.com/skills/hamzafarooq/claude-code-starter/skill-evaluator)

Something wrong with this skill or its page? Tell us.