watcher-hermes/hermes-skills/attention-variant-picker

Claude Code and agent skill by watcher-hermes/hermes-skills

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Pick a full / sliding-window / sparse / differential attention topology for a new model given context length, retrieval demands, and compute profile.

Install

npx vskill@latest install watcher-hermes/hermes-skills/attention-variant-picker

SKILL.md

The same SKILL.md was published earlier as watcher-hermes/hermes-skills/attention-variant-picker.

Attention Variant Picker

Help a developer choose and justify an attention topology for a new transformer, or for an existing one they're extending to longer context.

Inputs to gather

  1. Target context length at training and at inference (often different — many models train at 16K and extend at inference).
  2. Retrieval demand on a 1–5 scale: 1 = pure chat, 5 = needle-in-haystack / RAG / code with long repository context.
  3. Inference memory budget per-request KV cache tolerance (bytes per token per layer is the right unit).
  4. Training cost tolerance — training SWA from scratch is cheap; retrofitting differential attention into a pretrained model is expensive.
  5. Hardware target — Hopper+ has full FlashAttention-3, Ada has FA2, older GPUs are mask-limited.

Decision rules

  • Context ≤ 16K and retrieval ≤ 3: full attention with FlashAttention. Don't optimize prematurely.
  • Context 16–128K and retrieval ≤ 3: mixed SWA + global at 5:1, window 1024 (Gemma 3 shape). Keeps retrieval workable while collapsing KV.
  • Context > 128K: full SWA with a global layer every 4–6 layers, plus position interpolation / YaRN scaling (Lesson 04).
  • Retrieval = 5 and training budget allows: consider differential attention in the top 4 layers only (half the KV doubling, most of the sink-cancellation win).
  • You're shipping a public API: prefer stable patterns (full, SWA, Gemma-3 mix). Skip native-sparse / DIFF unless you have kernel engineers.
  • You can't change the base model: SWA can be retrofitted at inference via masking; differential and sparse can't.

Always flag

  • Pure-SWA models below 7B often lose measurably on reasoning benchmarks. Recommend against.
  • Window size < 512 is almost never right. Go bigger or use a different topology.
  • Differential attention reports in the paper are on small models (3–7B). Scale-up evidence is thin as of early 2026.
  • Every variant interacts with RoPE / YaRN scaling (Lesson 04). State the position scheme explicitly.

Output format

Return:

  1. Recommendation — a single named topology (e.g. "Gemma-3 mix, W=1024, 5:1 SWA:global").
  2. Justification — map each input to the decision rule above.
  3. KV cache estimate — at target context, in bytes per token per layer and GB at batch 1.
  4. Migration path — if the base model is already trained, how to retrofit.
  5. Known risks — which benchmarks / workloads might regress.

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

7-day trend score
5

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/watcher-hermes/hermes-skills/attention-variant-picker/badge)](https://verified-skill.com/skills/watcher-hermes/hermes-skills/attention-variant-picker)

Something wrong with this skill or its page? Tell us.