yzj623/pplab/autoattack-evaluation-protocol

Claude Code and agent skill by yzj623/pplab

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Evaluate robust accuracy with a fixed sequential AutoAttack-style attack protocol and per-attack accounting.

Install

npx vskill@latest install yzj623/pplab/autoattack-evaluation-protocol

SKILL.md

The same SKILL.md was published earlier as vectorspacelab/arex-skill/autoattack-evaluation-protocol.

AutoAttack Evaluation Protocol

Use this skill when converting attack outputs into robust accuracy under a fixed attack sequence. Do not use it to tune attacks based on model-specific failures; the order and attack definitions should be declared before evaluation.

Inputs

  • Examples and labels.
  • A logit function.
  • Ordered attack callables that return adversarial examples for a subset.

Outputs

  • Robust accuracy.
  • Remaining robust mask.
  • Per-attack success counts and evaluated counts.

Workflow

  1. Mark examples that are clean-correct.
  2. For each attack, pass only examples still robust.
  3. Update the robust mask when adversarial predictions change the label.
  4. Report final robust accuracy over all examples and per-attack accounting.

Validation

Run python tests/test_protocol.py or use the generated skill validator.

Limitations

This helper records the AutoAttack accounting pattern; it does not implement FAB or Square Attack internally.

Security

  • Security scanPassed, 90 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

7-day trend score
5

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/yzj623/pplab/autoattack-evaluation-protocol/badge)](https://verified-skill.com/skills/yzj623/pplab/autoattack-evaluation-protocol)

Something wrong with this skill or its page? Tell us.