amd-agi/geak/tuning-ck

Tuning Composable Kernel

Read ../tuning-core/SKILL.md first. Install the profiler per ../env-setup/ckprofiler_install.md — it is an apt package, about a minute, not a source build.

Like hipBLASLt, CK is a selection problem: CK ships pre-compiled instances and tuning means finding which one wins for your shape. Unlike hipBLASLt, the instance name is fully descriptive, so a CK sweep also tells you what shape of kernel your problem wants — useful even when you end up deploying through a different backend.

Scope, stated up front. This skill covers selection: ckProfiler races the 71 ops the profiler exposes (39 of them GEMM-family), and aiter's seven per-op CK tuners deploy the result. It does not cover CK's other tuning surface. tile_engine/ in the CK source is a JSON-declared config space (tile_m/n/k as {min,max,step}, warp_* as value lists) that generates and compiles instances that do not yet exist — the Triton model, in CK C++. If your shape's best kernel is not in the shipped library, selection cannot reach it and tile_engine is where you go. Nothing here describes that path; ../docs/coverage_gfx950.md §11 records it as an open gap rather than pretending it is covered.

Measured on gfx942 / MI300X and gfx950 / MI355X, ROCm 7.2.2, bf16. Numbers below are labelled per part; where only one is given it is gfx942. The apt package ships gfx950 instances, so nothing about the install changes between the two.

1. The interface is positional

ckProfiler takes positional arguments, not flags. Run ckProfiler <op> with nothing else and it prints its own legend — do that for any op you have not used lately, since the legend is authoritative for your build:

arg1: tensor operation (gemm: GEMM)
arg2: data type (0: fp32; 1: fp16; 2: bf16; 3: int8; 4: fp8)
arg3: matrix layout (0: A[m,k]*B[k,n]; 1: A[m,k]*B[n,k]; 2: A[k,m]*B[k,n]; 3: A[k,m]*B[n,k])
arg4: verification (0: no; 1: yes)
arg5: initialization (0: no init; 1: integer; 2: decimal)
arg6: print tensor value (0: no; 1: yes)
arg7: time kernel (0: no, 1: yes)
arg8 to 13: M, N, K, StrideA, StrideB, StrideC
arg14/15: warm-up cycles (default 1), iterations (default 10)

ckProfiler with no arguments at all lists every op — gemm, batched_gemm, grouped_gemm, fused variants like gemm_add_relu, conv, and more.

A 4096³ bf16 run with verification on, 5 warm-up and 20 iterations:

ckProfiler gemm 2 1 1 1 0 1 4096 4096 4096 4096 4096 4096 5 20

Two arguments are not optional in practice:

• arg4 verification = 1. An instance that is fast because it computes the wrong thing is

a real failure mode, and this is the cheapest place to catch it. Turn it off only for timing-sensitive reruns after correctness is established.

• arg7 time kernel = 1. Otherwise you get no timings.

The defaults for warm-up (1) and iterations (10) are too few to be stable — raise them, and read ../tuning-core/measurement.md on why one run is not a measurement.

arg3 layout is where results silently become incomparable. CK's layout codes describe which operands are transposed; a number from layout 1 cannot be compared against a number from layout 0. Match the layout to what your workload actually issues.

2. Read the instance name, not just the time

Output is one Perf: line per instance, then a best line:

Perf: 0.381103 ms, 360.634 TFlops, ... DeviceGemm_Xdl_CShuffle<MNKPadding, 256, 128, 128, 32, 8, 8, 32, 32, 2, 2, 8, 8, 1, 1> ... PipelineVersion: v2
Perf: 0.983496 ms, 139.745 TFlops, ... DeviceGemm_Xdl_CShuffle<MNKPadding, 128,  32, 128, 32, 8, 8, 32, 32, 1, 2, 8, 8, 1, 1> ... PipelineVersion: v2
 
Best Perf ... 0.251566 ms, 546.334 TFlops, 400.147 GB/s,
  DeviceGemm_Xdl_CShuffleV2<Default, 256, 256, 256, 32, 8, 8, 32, 32, 4, 4, 8, 8, 1, 1>
  LoopScheduler: Default, PipelineVersion: v1

7× spread between the best and worst instance on one shape (546 vs 140 TFLOPS). That range is the value of racing: picking an instance by intuition has a wide floor.

The same command on gfx950 / MI355X, 126 instances timed:

Best Perf ... 0.110352 ms, 1245.46 TFlops, 912.204 GB/s,
  DeviceGemm_Xdl_CShuffleV2<Default, 256, 256, 256, 32, 8, 8, 32, 32, 4, 4, 8, 8, 1, 1>
  LoopScheduler: Default, PipelineVersion: v1

The spread is the same order on both parts, so the argument for racing holds unchanged. The striking part is the last row: the winner is the same instance string, including padding strategy, tile, MFMA shape, pipeline and loop scheduler. Two parts with different CU counts and 2.5× the LDS agreed on which of 126 instances to pick.

Do not over-read that. It is one shape, and the hipBLASLt tuner's winner on the same shape did move between parts (a 224-deep tile to a 256-deep one). What it supports is the weaker and more useful claim in the next paragraph — the tile is evidence about the problem — rather than "CK winners are portable". Re-race per part; expect the answer to be recognisable.

The template parameters are the tuning knobs, made visible:

Plus PipelineVersion and LoopScheduler, which control software pipelining. Note the winner here is CShuffleV2 ... PipelineVersion: v1 — a different device op and pipeline from the v2 instances just below it, not merely a different tile. When you compare winners across shapes, compare all three: op variant, tile, pipeline.

This is why a CK sweep is worth running even if you deploy elsewhere: 256×256×256 winning by 1.5× over 256×128×128 on this shape is transferable evidence about what the problem wants, and it corroborates the macro-tiles chosen independently by the hipBLASLt and aiter tuners on the same shape.

2b. The candidate pool is not the same on both parts, and the gate is a string

"126 instances timed" is not a property of the op. CK's instance registration contains runtime device-name checks that add or remove instances before you ever see them. In library/src/tensor_operation_instance/gpu/:

add_device_operation_instances(instances, ..._comp_instances<GemmDefault>{});
 
if(ck::get_device_name() != "gfx950")
{
    add_device_operation_instances(instances, ..._comp_instances_part2<GemmDefault>{});
}

The part2 list is introduced in its header by the comment // instances not working on gfx950. Across the instance tree there are 166 such gates: 87 excluding on gfx950 and 79 adding only on gfx950. They are concentrated exactly where LLM serving lives — gemm_universal alone has 45 exclusion sites and 6 gfx950-only ones, plus gemm_universal_streamk (10), gemm_universal_reduce (8 / 4), batched_gemm (4 gfx950-only), grouped_gemm_tile_loop (4), and the three gemm_*fastgelu fusions (4 each).

Three consequences:

• An instance count is not comparable across parts. Fewer instances on gfx950 for a given

op may mean the deny-list fired, not that codegen produced less.

• A tile that won on gfx942 may not be in the pool on gfx950 at all. The excluded bf16

part2 lists contain the deep 256×256×224-class tiles; if you carry a gfx942 winner over and ckProfiler never reports it, check the deny-list before concluding the instance is slow.

• It is an exact string compare on `get_device_name()`, the same failure class as the

framework device-name split in ../tuning-in-vllm/ §1. Anything reporting a device name that is not literally gfx950 takes the other branch.

None of this is logged. The only way to see it is to read the registration file for your op.

2c. gemm_mx — and why it is the only route to MXFP8 on MI355

The microscaled formats are CDNA4-only and are the reason to care about gfx950 at all, so it matters that CK is where they are reachable. On the shipped images, MXFP8 has no aiter operator (../docs/coverage_gfx950.md §12): the MXFP8 Python surface exists in newer aiter source but not in either installed build. CK's gemm_mx is present and works.

It is a separate profiler op with its own positional list — the arg2 slot is the format, not a layout:

# MXFP4  (f4 -> f16), layout 1 = A[m,k] * B[n,k], verify on, 4096x4096x8192
ckProfiler gemm_mx 0 1 1 1 0 1 4096 4096 8192 -1 -1 -1 1 5 20 0
# MXFP8  (f8 -> f16)
ckProfiler gemm_mx 1 1 1 1 0 1 4096 4096 8192 -1 -1 -1 1 5 20 0
# MXFP8  (f8 -> bf16)
ckProfiler gemm_mx 2 1 1 1 0 1 4096 4096 8192 -1 -1 -1 1 5 20 0

arg2 is 0: f4->f16, 1: fp8->f16, 2: fp8->bf16. arg3 is the layout (0 MK_KN, 1 MK_NK, 2 MK_MFMA / B-preshuffled). Everything after that matches the ordinary gemm op.

Measured on gfx950, 4096·4096·8192, verification on:

Three things worth taking from that table.

The spread justifies racing. 2.3–2.8× between best and worst instance is the same order as the bf16 case in §2, so the argument for measuring rather than guessing survives into the MX formats.

The candidate pool is small and asymmetric. Eleven FP4 instances against five FP8. This is the apt-packaged profiler; the CK source tree carries 67 MX instances across six type combinations (f4/f8/f6/bf6/bf8), so what you can race is a fraction of what CK can build, and gemm_mx is gated to gfx95 at build time. If you need an instance that is not in the shipped set, that is a build, not a flag.

The instance names confirm the format, and are worth reading for that alone. The winner prints as ck::f4x2_pk_t (two FP4 packed per byte) or ck::f8_ocp_t — the OCP dialect, not FNUZ, which is the gfx950 side of the inversion in ../tuning-core/measurement.md. Every MX instance also prints ScaleBlockSize: 32, which is the one number the whole format hangs on: one E8M0 exponent per 32 elements along K. A split-K or block-K choice that does not divide 32 cuts a scale group, and the result is finite, plausible and wrong — the same trap the FP4 corpus cases guard against with an explicit NUM_KSPLIT constraint.

These are race numbers, so read them under ../tuning-core/measurement.md Rule 6b: on gfx950 a back-to-back race inflates. The 2.3–2.8× ranking is far outside that error, but the absolute TFLOPS are not deploy-grade until replayed.

3. ckProfiler does not change what your framework runs

A ckProfiler winner is a fact about CK, not a change to your workload. Nothing in your serving path consults ckProfiler output.

Use it for:

• Establishing a ceiling — what could a CK path reach for this shape, before investing in

wiring one up.

• Backend triage — is CK even competitive here versus hipBLASLt or Triton? Run the same

shape through each and compare. On 4096³ bf16 CK's best was 546 TFLOPS against hipBLASLt's 633 on gfx942, and 1245 against 1241 on gfx950 — worth knowing before spending a day on the CK route. Note that the gap closed to nothing on the newer part, so this is a question to re-ask per part rather than a settled ranking.

• Sanity-checking a suspicious result from a higher-level tuner.

To actually change dispatch, go through aiter's per-op CK tuners, which write config CSVs the runtime reads. There are seven, and which one you want is set by the dtype pair: