nickgallick/perlantir-fleet/benchmark-export-packaging

Claude Code and agent skill by nickgallick/perlantir-fleet

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Install

npx vskill@latest install nickgallick/perlantir-fleet/benchmark-export-packaging

SKILL.md

The same SKILL.md was published earlier as nickgallick/perlantir-fleet/benchmark-export-packaging.

Benchmark Export Packaging — Skill 88

Purpose

Package challenge data for the Bouts Benchmark API and data licensing. AI labs purchase access — the export must be clean, structured, and valuable.

Per-Challenge Export Structure

json
{
"challenge_meta": {
"family": "string",
"format": "string",
"weight_class": "string",
"difficulty_profile": {},
"cdi_grade": "string",
"total_attempts": "integer"
},
"aggregate_results": {
"score_distribution": {
"mean": "float",
"median": "float",
"std_dev": "float",
"p10": "float",
"p90": "float"
},
"component_averages": {
"objective": "float",
"process": "float",
"strategy": "float",
"recovery": "float",
"integrity": "float (avg adjustment)"
},
"failure_archetype_distribution": {
"archetype_name": "float (0-1 proportion)"
},
"by_model_family": {
"family_name": { "mean": "float", "n": "integer" }
}
}
}

What's NEVER in the Export

  • ❌ Individual agent submissions (code, deliverables)
  • ❌ Exact challenge instances (briefings, codebases, test suites)
  • ❌ Individual agent identities or ELO ratings
  • ❌ Judge prompts or exact scoring formulas
  • ❌ Hidden test logic
  • ❌ Reference solutions

What IS in the Export

  • ✅ Aggregate statistics per challenge family/format/weight class
  • ✅ Failure archetype distributions
  • ✅ Score distributions by model family
  • ✅ Component score breakdowns
  • ✅ CDI trends over time
  • ✅ Headline insights: "AI agents in 2026 pass 72% of static tests but only 31% of adversarial tests"

Export Tiers

TierAudienceContent Depth
PublicCommunity, pressHeadline stats, quarterly Bouts Index
StandardRegistered labsPer-family breakdowns, archetype distributions
PremiumPaying enterprise/labsModel family comparisons, trend data, private lane results

The Bouts AI Agent Index (Public Report)

Quarterly publication:

  • Overall agent capability trends
  • Hardest challenge families
  • Most common failure archetypes
  • Model family performance comparison (anonymized)
  • CDI trends (are challenges getting better at discriminating?)
  • Headline insights for press and social media

Integration Points

  • CDI (Skill 46): CDI grades included in all exports
  • Failure Archetypes (Skill 48): Archetype distributions are core export data
  • Defensibility Reporting (Skill 57): Export methodology documented for trust
  • Challenge Economy (Skill 58): Licensing value drives export packaging decisions

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

7-day trend score
5

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/nickgallick/perlantir-fleet/benchmark-export-packaging/badge)](https://verified-skill.com/skills/nickgallick/perlantir-fleet/benchmark-export-packaging)

Something wrong with this skill or its page? Tell us.