lyndonkl/claude/ml-primitive-decoder

ML Primitive Decoder

Table of Contents

1. Workflow
2. The Primitive Catalog
3. Decoding Patterns
4. Ablation Thought Experiment
5. Common Patterns
6. Guardrails
7. Quick Reference

Most ML constructs that look opaque are actually a short stack of well-understood linear algebra primitives composed in a specific way. Attention is dot product + softmax + weighted sum. Layer norm is recenter + rescale + learnable affine. Diffusion is noise injection + denoising regression repeated. This skill names the primitives, then shows why their composition produces the behavior the construct is famous for.

The signature deliverable is: construct = primitive₁ + primitive₂ + primitive₃, and *that's why* it does X.

Quick example (Attention):

Construct: Self-attention layer, output = softmax(QKᵀ/√d) · V.

Primitives:

1. Linear projections (Q, K, V each = a different learnable linear layer applied to the same input).

2. Dot product (QKᵀ measures pairwise similarity between query and key vectors).

3. Scaled softmax (divides scores by √d to keep variance bounded, then row-normalizes into mixing weights).

4. Weighted sum (multiplying by V is a per-row weighted average over value vectors).

Why it works: Each token learns to advertise (K), ask (Q), and deliver (V). The dot product matches asks to advertisements; softmax turns raw matches into selection weights; the weighted sum is the selected content. The whole layer is content-addressable memory implemented in pure linear algebra.

Ablation:

- Remove softmax → linear combination, no selectivity (just averaging everything).

- Remove √d → softmax saturates as d grows, gradients vanish.

- Tie Q = K → tokens can only ask questions about themselves; no cross-token information transfer.

Workflow

Copy this checklist and track your progress:

Decoding Progress:
- [ ] Step 1: Identify the construct and write its formula
- [ ] Step 2: List the primitives it's composed of
- [ ] Step 3: Explain what each primitive contributes
- [ ] Step 4: Show how composition produces the famous behavior
- [ ] Step 5: Run the ablation thought experiment (what breaks if you remove each piece?)
- [ ] Step 6: Verify with a tiny worked example or invitation

Step 1: Identify the construct and write its formula

Pin down what you're decoding. Get the formula in front of you (and the user). If the user asks "why does attention work" without specifying, it's almost always self-attention — but check. Multi-head attention, cross-attention, and flash attention are different decompositions.

For a catalog of common ML constructs and their formulas, see resources/constructs.md.

Step 2: List the primitives it's composed of

The primitives are a small fixed set (see The Primitive Catalog below). Decompose the formula into a list of these. Most constructs are 2-5 primitives.

The discipline: name only the load-bearing primitives. Trivial reshapes, broadcasts, and identity passes don't count. If a primitive doesn't change the behavior of the construct, skip it.

Step 3: Explain what each primitive contributes

For each primitive, write one sentence saying what it does in this construct specifically — not in general. "Dot product measures alignment" is too generic. "Dot product, applied to (Q, K) pairs, scores how well each token's question matches each other token's advertisement" is specific.

Step 4: Show how composition produces the famous behavior

This is the payoff. The construct is famous for some behavior — attention does content-addressable retrieval; layer norm stabilizes training; softmax produces a sharp-or-smooth distribution. Show how the combination of primitives produces that behavior. The composition is where the magic isn't.

A good Step 4 reads like: "Primitive A produces sub-effect 1. Primitive B turns sub-effect 1 into sub-effect 2. Primitive C turns sub-effect 2 into the famous behavior. Each step is mechanical; the famous behavior emerges from the chain."

Step 5: Run the ablation thought experiment

For each primitive, ask: "what would break if we removed this?" The answers are the highest-information part of the decoding — they tell the learner why each piece is there, which is much harder to forget than the formula.

See Ablation Thought Experiment below for the structure. For ablation tables per construct, see resources/ablations.md.

Step 6: Verify with a tiny worked example or invitation

Either compute one tiny instance of the construct end-to-end (a 3-token attention, a 2-feature layer norm) showing each primitive's intermediate output — or invite the user to do so. The worked example is what makes the decomposition feel real, not just declarative.

For tiny worked examples per construct, hand off to the worked-example-walkthrough skill.

The Primitive Catalog

Almost every ML construct is built from a small set of these:

Whenever you decode a construct, the primitives list should come from this catalog. If you find yourself naming something that isn't here, either it's a more complex construct (decompose further) or it's a new primitive worth adding.

Decoding Patterns

Pattern A: The pipeline construct

The construct is a sequence of primitives applied in order. Decode as: input → P₁ → intermediate → P₂ → ... → output. Examples: attention block, transformer FFN, layer norm.

Pattern B: The decorated construct

The construct is a base primitive wrapped with normalization, residuals, or activations. Decode as: base + decorator₁ + decorator₂. Examples: a transformer block (attention + residual + LN + FFN + residual + LN), ResNet block.

Pattern C: The objective construct

The construct is a loss function. Decode as: similarity measure + normalization + reduction. Examples: cross-entropy (log + sum), contrastive loss (similarity + softmax + cross-entropy).

Pattern D: The iterative construct

The construct is a single step repeated many times. Decode the step, then explain the dynamics of repetition. Examples: gradient descent (gradient + step), diffusion (noise + denoise step), RNN (state update step).

For one full decode per pattern, see resources/constructs.md.

Ablation Thought Experiment

For every decoded construct, run an ablation table. For each primitive, answer: "what would break if you removed it (or replaced it with the simplest possible thing)?"

This is the most pedagogically valuable part of the skill. It forces the user to see why each piece is there.

Format:

Example (Attention):

For ablation tables for many constructs, see resources/ablations.md.

Common Patterns

When the user asks "why does X work?"

Always run the full workflow. The "why" is in Step 4 (composition) and Step 5 (ablation), not in Step 2 (primitive list).

When the user asks "what is X doing?"

Steps 1-3 may be enough. Skip ablation if the user just wants a quick read.

When the construct is opaque even to you

Don't fake it. Decode what you understand; flag what you don't. "I can decode the attention part but I'm not sure why this paper added the gating; let me look it up" is a much better response than confident-sounding nonsense.

When the construct is novel

Decompose into the catalog of primitives anyway. Truly novel ML constructs are rare; most are reshufflings of the catalog. If something genuinely doesn't fit, name it as a new primitive and explain why.

Guardrails

• Don't list primitives without saying what they do *in this construct*. Generic descriptions ("softmax produces a probability distribution") are worth less than construct-specific ones ("softmax converts the row of dot product scores into mixing weights for the value vectors").
• Don't skip the ablation. It's the part that builds durable understanding. The user who can answer "what would happen if you removed the softmax from attention?" understands attention; the user who can only recite the formula does not.
• Don't bury the punchline. Step 4 — "and that's why it does X" — should be loud and short. If the punchline is buried in paragraph 4, the reader missed it.
• Don't decode further than necessary. Linear projection is a primitive. You don't need to decode it into "matrix multiply, which is itself a sum of weighted columns…" unless the user asks. Decompose to the level the user needs and stop.
• Don't conflate the construct with its claimed *purpose*. Attention isn't "how transformers attend to relevant context" — that's its emergent behavior. The construct is softmax(QKᵀ/√d)V; the behavior emerges from training. Be precise about what the math forces vs. what learning produces.

Quick Reference