islinxu/eval-skills/guides

Claude Code and agent skill by islinxu/eval-skills

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Install

npx vskill@latest install islinxu/eval-skills/guides

SKILL.md

Creating a Custom Skill

This guide covers how to create a Skill that works with eval-skills.

Skill Structure

Every Skill needs a skill.json metadata file:

json
{
  "id": "my_skill",
  "name": "My Custom Skill",
  "version": "1.0.0",
  "description": "What this skill does",
  "tags": ["category1", "category2"],
  "inputSchema": {
    "type": "object",
    "properties": {
      "query": { "type": "string" }
    },
    "required": ["query"]
  },
  "outputSchema": {
    "type": "object",
    "properties": {
      "result": { "type": "string" }
    }
  },
  "adapterType": "subprocess",
  "entrypoint": "python3 skill.py",
  "metadata": {
    "author": "Your Name",
    "license": "MIT"
  }
}

Adapter Types

Subprocess (JSON-RPC 2.0)

The simplest way to create a Skill. Your script reads JSON-RPC from stdin and writes to stdout.

Protocol:

stdin:  {"jsonrpc": "2.0", "method": "invoke", "params": {...}, "id": 1}
stdout: {"jsonrpc": "2.0", "result": {...}, "id": 1}

Python Template:

python
#!/usr/bin/env python3
import json
import sys
def invoke(params):
    query = params.get("query", "")
    # Your logic here
    return {"result": f"Processed: {query}"}
def healthcheck(params):
    return {"status": "healthy"}
def main():
    raw = sys.stdin.read()
    request = json.loads(raw)
    method = request.get("method", "")
    params = request.get("params", {})
    req_id = request.get("id", 1)
    handlers = {"invoke": invoke, "healthcheck": healthcheck}
    handler = handlers.get(method)
    if handler:
        result = handler(params)
        response = {"jsonrpc": "2.0", "result": result, "id": req_id}
    else:
        response = {
            "jsonrpc": "2.0",
            "error": {"code": -32601, "message": f"Unknown method: {method}"},
            "id": req_id,
        }
    print(json.dumps(response))
if __name__ == "__main__":
    main()
HTTP

For Skills exposed as HTTP services.

skill.json:

json
{
  "adapterType": "http",
  "entrypoint": "http://localhost:3000/api/invoke"
}

adapter.config.json:

json
{
  "type": "http",
  "baseUrl": "http://localhost:3000",
  "authType": "bearer",
  "authTokenEnvKey": "MY_SKILL_API_KEY",
  "timeoutMs": 15000
}

The HTTP adapter sends:

POST /api/invoke
Content-Type: application/json
 
{
  "skillId": "my_skill",
  "version": "1.0.0",
  "input": { "query": "hello" }
}

Using Templates

Generate a skeleton with:

bash
# Python subprocess skill
eval-skills create --name my_skill --from-template python_script
 
# HTTP skill
eval-skills create --name my_api --from-template http_request
 
# MCP tool skill
eval-skills create --name my_mcp --from-template mcp_tool

Testing Your Skill

  1. Create evaluation tasks in tests/basic.eval.json:
json
[
  {
    "id": "test_001",
    "description": "Basic test",
    "inputData": { "query": "hello" },
    "expectedOutput": { "type": "contains", "keywords": ["hello"] },
    "evaluator": { "type": "contains" },
    "timeoutMs": 10000
  }
]
  1. Run evaluation:
bash
eval-skills eval --skills ./skills/my_skill/ --tasks ./skills/my_skill/tests/basic.eval.json

Best Practices

  1. Always define input/output schemas — helps with validation and documentation
  2. Add meaningful tags — makes your Skill discoverable
  3. Handle errors gracefully — return structured error responses
  4. Keep latency low — aim for P95 < 5 seconds
  5. Write comprehensive evaluation tasks — cover edge cases
  6. Version your Skill — use semver for tracking changes

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

7-day trend score
0

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/islinxu/eval-skills/guides/badge)](https://verified-skill.com/skills/islinxu/eval-skills/guides)

Something wrong with this skill or its page? Tell us.