yzj623/pplab/attention-path-expansion
Claude Code and agent skill by yzj623/pplab
- community
Checking that the SKILL.md file is still in the repository.
Compute frozen-attention direct, head, and virtual-head path contributions for attention-only circuits.
Install
npx vskill@latest install yzj623/pplab/attention-path-expansionSKILL.md
The same SKILL.md was published earlier as yzj623/pplab/attention-path-expansion.
Attention Path Expansion
Use this skill when you have fixed or frozen attention patterns and want to decompose attention-only logits into additive circuit paths. It is suited for toy transformers, ablations, and mechanism checks inspired by Transformer Circuits. Do not use it for full transformers with changing layer normalization or MLP paths unless those parts are explicitly separated.
Inputs
- Token ids and an embedding matrix.
- Unembedding matrix.
- One or more attention patterns shaped destination-position by source-position.
- Matching OV matrices shaped model-by-model.
Outputs
- Direct residual-to-logit contribution.
- First-order per-head contributions.
- Optional virtual-head contribution for a two-head sequence.
- Maximum consistency error against explicit frozen-attention forward computation.
Workflow
- Embed tokens into residual states.
- Compute direct logits through the unembedding.
- Apply each head as
attention @ residual @ OVand unembed the result. - For two sequential heads, compute the virtual contribution
A2 @ A1 @ residual @ OV1 @ OV2. - Compare the expanded sum with an explicit sequential frozen-attention forward pass.
Validation
Run the deterministic tests or validate with the module-to-skill tree validator.
Limitations
The script implements the paper's simplified attention-only setting with supplied attention patterns. It intentionally does not implement softmax QK score construction.
Security
- Security scanPassed, 90 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
Popularity
- 7-day trend score
- 5
Refreshed 11 Oct 2026, a month ago.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/yzj623/pplab/attention-path-expansion)