dixiyao/fot/pinchbench

Claude Code and agent skill by dixiyao/fot

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Run PinchBench benchmarks to evaluate OpenClaw agent performance across real-world tasks. Use when testing model capabilities, comparing models, submitting benchmark results to the leaderboard, or checking how well your OpenClaw setup handles calendar, email, research, coding, and multi-step workflows.

Install

npx vskill@latest install dixiyao/fot/pinchbench

SKILL.md

PinchBench Benchmark Skill

PinchBench measures how well LLM models perform as the brain of an OpenClaw agent. Results are collected on a public leaderboard at pinchbench.com.

Prerequisites

  • Python 3.10+
  • uv package manager
  • OpenClaw instance (this agent)

Quick Start

bash
cd <skill_directory>
 
# Run benchmark with a specific model
uv run benchmark.py --model anthropic/claude-sonnet-4
 
# Run only automated tasks (faster)
uv run benchmark.py --model anthropic/claude-sonnet-4 --suite automated-only
 
# Run specific tasks
uv run benchmark.py --model anthropic/claude-sonnet-4 --suite task_calendar,task_stock
 
# Skip uploading results
uv run benchmark.py --model anthropic/claude-sonnet-4 --no-upload

Available Tasks (23)

TaskCategoryDescription
task_sanityBasicVerify agent works
task_calendarProductivityCalendar event creation
task_stockResearchStock price lookup
task_blogWritingBlog post creation
task_weatherCodingWeather script
task_summaryAnalysisDocument summarization
task_eventsResearchConference research
task_emailWritingEmail drafting
task_memoryMemoryContext retrieval
task_filesFilesFile structure creation
task_workflowIntegrationMulti-step API workflow
task_clawdhubSkillsClawHub interaction
task_skill_searchSkillsSkill discovery
task_image_genCreativeImage generation
task_humanizerWritingText humanization
task_daily_summaryProductivityDaily digest
task_email_triageEmailInbox triage
task_email_searchEmailEmail search
task_market_researchResearchMarket analysis
task_spreadsheet_summaryAnalysisSpreadsheet analysis
task_eli5_pdf_summaryAnalysisPDF simplification
task_openclaw_comprehensionKnowledgeOpenClaw docs comprehension
task_second_brainMemoryKnowledge management

Command Line Options

OptionDescription
--modelModel identifier (e.g., anthropic/claude-sonnet-4)
--suiteall, automated-only, or comma-separated task IDs
--output-dirResults directory (default: results/)
--timeout-multiplierScale task timeouts for slower models
--runsNumber of runs per task for averaging
--no-uploadSkip uploading to leaderboard
--registerRequest new API token for submissions
--upload FILEUpload previous results JSON

Token Registration

To submit results to the leaderboard:

bash
# Register for an API token (one-time)
uv run benchmark.py --register
 
# Run benchmark (auto-uploads with token)
uv run benchmark.py --model anthropic/claude-sonnet-4

Results

Results are saved as JSON in the output directory:

bash
# View task scores
jq '.tasks[] | {task_id, score: .grading.mean}' results/0001_anthropic-claude-sonnet-4.json
 
# Show failed tasks
jq '.tasks[] | select(.grading.mean < 0.5)' results/*.json
 
# Calculate overall score
jq '{average: ([.tasks[].grading.mean] | add / length)}' results/*.json

Adding Custom Tasks

Create a markdown file in tasks/ following TASK_TEMPLATE.md. Each task needs:

  • YAML frontmatter (id, name, category, grading_type, timeout)
  • Prompt section
  • Expected behavior
  • Grading criteria
  • Automated checks (Python grading function)

Leaderboard

View results at pinchbench.com. The leaderboard shows:

  • Model rankings by overall score
  • Per-task breakdowns
  • Historical performance trends

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

Repository stars
111
Repository forks
2
7-day trend score
24

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/dixiyao/fot/pinchbench/badge)](https://verified-skill.com/skills/dixiyao/fot/pinchbench)

Something wrong with this skill or its page? Tell us.