vectorspacelab/arex-skill/accuracy-pair-protocol
Claude Code and agent skill by vectorspacelab/arex-skill
- community
Checking that the SKILL.md file is still in the repository.
Validate paired ID/OOD accuracy records before fitting accuracy-on-the-line calibration experiments.
Install
npx vskill@latest install vectorspacelab/arex-skill/accuracy-pair-protocolSKILL.md
The same SKILL.md was published earlier as vectorspacelab/arex-skill/accuracy-pair-protocol.
Accuracy Pair Protocol
Use this skill when a recovery or benchmark script needs to build the paired in-distribution and out-of-distribution accuracy panel used by Accuracy on the Line. Do not use it to fit the line or decide whether the proxy passes; it only owns record validation and deterministic ordering.
Inputs
- A JSON list of objects with
model_id,id_accuracy, andood_accuracy. - Optional metadata fields such as architecture, training duration, or shift name.
Outputs
- A JSON object containing validated
records,count,id_range,ood_range, andprovenance. - Validation errors for missing fields, duplicate model ids, nonnumeric accuracies, or values outside
[0, 1].
Workflow
- Parse records from JSON or receive them as Python dictionaries.
- Require one paired ID and OOD accuracy for every model id.
- Enforce numeric ranges and unique model ids.
- Sort records by
model_idso downstream line fitting is reproducible. - Preserve optional metadata without adding predictions or pass/fail decisions.
Validation
Run python scripts/accuracy_pairs.py --input tests/fixtures/valid_pairs.json or validate the tree with the Distiller validate_skill_tree.py --run-tests command.
Limitations
This skill does not download datasets, train models, fit the calibration line, or evaluate a recovery target. Synthetic records must be labeled by the caller as proxy evidence.
Security
- Security scanPassed, 90 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingskills/task-oriented/PaperBench/skills/lca-on-the-line/skill/accuracy_on_the_line/accuracy_pair_protocol/SKILL.md
Popularity
- 7-day trend score
- 0
Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/vectorspacelab/arex-skill/accuracy-pair-protocol)