yzj623/pplab/autoattack-evaluation-protocol
Claude Code and agent skill by yzj623/pplab
- community
Checking that the SKILL.md file is still in the repository.
Evaluate robust accuracy with a fixed sequential AutoAttack-style attack protocol and per-attack accounting.
Install
npx vskill@latest install yzj623/pplab/autoattack-evaluation-protocolSKILL.md
The same SKILL.md was published earlier as vectorspacelab/arex-skill/autoattack-evaluation-protocol.
AutoAttack Evaluation Protocol
Use this skill when converting attack outputs into robust accuracy under a fixed attack sequence. Do not use it to tune attacks based on model-specific failures; the order and attack definitions should be declared before evaluation.
Inputs
- Examples and labels.
- A logit function.
- Ordered attack callables that return adversarial examples for a subset.
Outputs
- Robust accuracy.
- Remaining robust mask.
- Per-attack success counts and evaluated counts.
Workflow
- Mark examples that are clean-correct.
- For each attack, pass only examples still robust.
- Update the robust mask when adversarial predictions change the label.
- Report final robust accuracy over all examples and per-attack accounting.
Validation
Run python tests/test_protocol.py or use the generated skill validator.
Limitations
This helper records the AutoAttack accounting pattern; it does not implement FAB or Square Attack internally.
Security
- Security scanPassed, 90 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingbasicagent-skill/skill_library/paperbench/robust-clip/skill/autoattack_apgd/autoattack_evaluation_protocol/SKILL.md
Popularity
- 7-day trend score
- 5
Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/yzj623/pplab/autoattack-evaluation-protocol)