amd-agi/geak/roofline

SKILL — roofline headroom analysis

Turn the standardized Top-N into a per-kernel answer to: "how much of this hardware's ceiling is this kernel already getting, and therefore how much is left to win?"

You (an agent) can execute this skill by reading this file. roofline_tools.py next to it holds mechanical primitives (peak-table parsing, counter parsing, unit math) — use them if they import, do the arithmetic yourself if they don't. A broken helper must not disable this skill; a broken skill must not fail the run.

0. The one doctrine that matters

roofline_pct measures how well the kernel executes its current algorithm's byte/FLOP budget.

It says nothing about whether that budget is necessary.

A kernel at 90% of the memory roofline is not "done" — it is done with kernel micro-tuning. The remaining lever is to move fewer bytes. Never read a high roofline_pct as "stop optimizing this kernel", especially when it is the largest consumer of GPU time. See §7.

Corollary: this skill is advisory. It reorders and annotates. pct_gpu_time remains the primary key and the measurement remains the judge. This skill may never prune a candidate.

1. Inputs

• profile/round_<R>/profile_topN.json — the standardized Top-N (required).
• env_report.json — gfx, structured device_target, physical_cu_count,

model_arch_class, model_dtype, workload (required).

• The model's config.json — layer count, expert count, hidden/intermediate sizes, head counts

(optional but needed for a good MoE/attention byte model).

• peaks.md — hardware denominators, keyed by gfx.
• Optionally, captured shapes from the Kernel Extractor and/or rocprofv3 counters (stage B/C, §5).

2. Output artifact

Write profile/round_<R>/profile_roofline.json and a human-readable profile_roofline.md beside it.

{
  "skill": "roofline", "skill_version": "1",
  "gfx": "gfx950",
  "peaks": { "hbm_bw_bytes_s": 8.0e12, "flops": {"fp8": 5.0e15},
             "source": "table|derived", "confidence": "high|low" },
  "stage": "A|B|C",
  "entries": [{
    "name": "...", "short_name": "...",
    "pct_gpu_time": 26.45,              // COPIED verbatim from the Top-N, never recomputed
    "regime": "decode|prefill",
    "op_class": "gemm|moe|attn|elementwise|unknown",
    "modeled": true,
    "t_ms": 0.049471,                   // per-launch time for this regime
    "launches_per_step": 80,
    "bytes_est": 0, "flops_est": 0,
    "achieved_bw_bytes_s": 0, "achieved_flops": 0,
    "hbm_util": 0.88, "compute_util": 0.01,   // achieved/peak on each axis; the pair decides bound_type
    "arithmetic_intensity": 4.6, "ridge_point": 625.0,
    "bound_type": "memory|compute|latency|unknown",  // latency = neither roof near its ceiling
    "roofline_pct": 0.88,               // achieved / peak on the AI-selected roof
    "target_eff": 0.90,
    "attainable_speedup": 1.023,
    "expected_e2e_gain_pct": 0.59,
    "headroom_class": "underperforming|moderate|saturated|unknown",
    "confidence": "low|medium|high",
    "suspect": false,
    "byte_reduction_levers": ["..."],   // populated only when headroom_class=saturated
    "notes": "assumptions made, what would sharpen this"
  }],
  "ranking_by_pct": ["..."],            // BOTH rankings are emitted, side by side
  "ranking_by_expected_gain": ["..."],
  "degraded": [ {"name": "...", "reason": "..."} ],
  "skill_errors": []
}

ranking_by_pct and ranking_by_expected_gain are both emitted deliberately. The consumer must see the disagreement rather than a single blended number that hides it.

3. Procedure

1. Scope to the head. Analyse ONLY entries with pct_gpu_time >= HEAD_THRESHOLD_PCT (default 5),

biggest first, capped at ~8. Below that bar the Amdahl ceiling is under the noise band no matter what the roofline says, so a headroom estimate cannot change a decision — modelling those kernels only adds failure modes. Skipped entries are absent from the artifact; they are NOT degraded[] (a kernel too small to matter is not a modelling failure and must not read as one).

1. Resolve peaks from peaks.md via

resolve_peaks(gfx, product=env_report.device_target). device_target=unknown (every non-R9700 card, MI300/MI355 included) is no product constraint: the ISA-keyed tables still resolve. Client RDNA4 numeric peaks exist only for product r9700. A bare gfx1201 (or any other gfx120x SKU) is a hard unknown: do not derive a numeric denominator, emit unknown headroom, and do not rank on roofline. gfx125x is CDNA5, not RDNA4 — it must not use that gate. Other unknown architectures may derive from device props with peaks.confidence="low" (§6 L1). Use peak_flops_for() canonical dtype keys; a missing dtype is unknown, never the table maximum.

1. For each selected entry, pick the e2e-critical regime — the one carrying the launches

(serving.n_decode_steps vs n_prefill_steps; a decode-dominated run means decode). Use that regime's base_latency_ms as t_ms.

1. Classify op_class from name/classification/shapes.
2. Apply the §4 byte/FLOP model for that class. Cannot model it → §6 L2 (degrade this entry only).
3. Compute:

``` achieved_bw = bytes_est / t achieved_flops = flops_est / t hbm_util = achieved_bw / peak_bw compute_util = achieved_flops / peak_flops AI = flops_est / bytes_est ridge_point = peak_flops / peak_bw

# AI picks which roof the kernel walks TOWARD; roofline_pct is measured on that roof. roof_axis = "compute" if AI > ridge_point else "memory" roofline_pct = compute_util if roof_axis == "compute" else hbm_util

# But which roof actually BINDS is decided by utilization, not by AI alone. A small AI does NOT # by itself mean memory-bound — that is the most common mislabel. If neither roof is near its # ceiling (both utils < 0.60) and the launch is above the dispatch floor, the kernel is # LATENCY / occupancy-bound, not bandwidth- or compute-bound. bound_type = "latency" if (hbm_util < 0.60 and compute_util < 0.60) else roof_axis

attainable_speedup = max(1.0, target_eff / roofline_pct) expected_e2e_gain_pct = pct_gpu_time × (1 − 1/attainable_speedup) `` A latency-bound kernel **still gets a headroom verdict** (its roofline_pct on the AI-selected roof is real, and target_eff already prices in the occupancy penalty for irregular classes like paged attention) — so a low-utilization head still ranks by its headroom. What changes is the **lever**: latency-bound underperformance is fixed by occupancy / shorter dependency chains / fusion, **not** by byte reduction (§7.1). Byte reduction only helps a genuinely bandwidth-bound (high-hbm_util`) head.

1. Classify headroom, banded against target_eff (not against the raw roofline — what matters is

the distance to what a good implementation of this class can realistically reach): roofline_pct ≥ 0.9×target_eff → saturated; ≥ 0.6×target_eff → moderate; else → underperforming; unmodelled or low-confidence → unknown. *88% against a 0.90 target is saturated, not "nearly there" — tuning has nothing left to give.*

Two outcomes must produce NO verdict (headroom_class: "unknown"), because in each the ratio is not evidence about the kernel:

• Dispatch-bound — the per-launch time is within launch-overhead scale (~5 µs), so the launch is

timed by dispatch, not by its transfer or its math. Emit bound_type: "latency"; the lever is fusion / graph capture, not kernel tuning. Typical of tiny high-call-count kernels. This is the no-verdict sub-case of latency-bound — distinct from the general latency-bound kernel in step 5 (low utilization but well above the dispatch floor), which keeps its verdict because it is doing real work and has recoverable occupancy/dependency headroom.

• Infeasible — roofline_pct outside (0,1]. That is the byte/FLOP model being wrong, not the

kernel being at the wall. A clamped 100% must NEVER be reported as `saturated` — that turns a modelling failure into a routing decision. A compute-axis ratio above 1.0 is most often an unvalidated peak (the BF16 MFMA microbench commonly reads ~2× low; see §4). See §6 L3.

bound_type is a CLOSED set: memory | compute | latency | unknown. If none fits, emit unknown — never invent a category the consumer has no routing rule for.

1. Sanity-check (§6 L3), emit both rankings, write the artifact.

Per-launch, not aggregate. Compare bytes for ONE launch against ONE launch's base_latency_ms. If a logical op is split across several launches (e.g. a fused-MoE layer issuing a stage-1 and a stage-2 kernel), sum the launches for one logical unit and compare against the summed time. Getting this factor wrong is the single most common way to produce a nonsense roofline_pct — state in notes which unit you used.

4. Byte / FLOP models by op class

Weight bytes use the weight dtype (fp8 = 1 B/elem, bf16 = 2 B). Activation bytes use the activation dtype. Only count HBM traffic — a tensor re-read within one launch and small enough to sit in L2 (l2_bytes) counts once.

Trust the memory axis over the compute axis, especially at decode. The compute peaks in peaks.md are validated for fp8, but empirical MFMA peaks are not always right — the BF16 microbench commonly reads ~2× low, which makes a BF16 compute_util read ~2× high (and can push roofline_pct above 1.0, where §6 L3 catches it as suspect). A decode workload is memory-bound anyway, so prefer hbm_util; only rank on a compute-axis roofline_pct after the peak for that dtype has been validated (§8 rule: BF16 and FP16 MFMA run at the same rate, so those two peaks must be equal — if they are not, the peak is mis-calibrated and the compute-axis number is not usable).

dense GEMM [M,K]×[K,N]

flops = 2·M·N·K
bytes = M·K·a + K·N·w + M·N·a            # A + B + C

MoE / grouped expert GEMM

The decisive question is how many expert weights are streamed, which dominates at decode.

pairs        = M · top_k
experts_hit  = E · (1 − (1 − 1/E)^pairs)      # expected distinct experts touched
flops = 2 · pairs · (per-expert MAC count for this stage)
bytes ≈ experts_hit · (per-expert weight elems) · w   + activations

If the implementation streams all E experts regardless of routing, use E instead of experts_hit — and note that the difference between the two IS a byte-reduction lever (§7). When unsure which the kernel does, compute both, report the experts_hit figure, and put the all-expert figure in notes.

Feasibility rule (general, applies to every op class with more than one plausible byte model). A byte estimate implying a rate above peak is refuted by the measurement itself. When you have several candidate models, pick the largest one that stays feasible (bytes ≤ peak_bw × t) and say which you used. If every candidate is infeasible, the class model is wrong: emit no verdict (§6 L3) rather than clamping to 100% and calling it saturated.

attention (paged decode)

KV traffic dominates; Q and the output are negligible at decode.

bytes ≈ batch · seq_len · n_kv_heads · head_dim · 2 (K and V) · kv_dtype_bytes
flops ≈ 2 · batch · n_q_heads · seq_len · head_dim · 2 (QK^T and PV)

seq_len is the current average context, not the max — for an isl/osl workload sampled mid-run, isl + osl/2 is a reasonable estimate. Say so in notes; it is a real source of error.

elementwise / norm / quant

bytes = (input elems · in_dtype) + (output elems · out_dtype)
flops = small — assume memory-bound unless clearly otherwise

unknown

Do not guess. Emit modeled: false and degrade this entry (§6 L2).

5. Confidence stages — the estimate sharpens as the run proceeds