matlab/matlab-agentic-toolkit/matlab-optimize-gpu-codegen

Optimize MATLAB for GPU Code Generation

Iteratively optimize a MATLAB design file for GPU Coder: compile, benchmark, apply structural optimizations, profile with gpuPerformanceAnalyzer, fix diagnostics, and verify numerical equivalence at every step.

When to Use

• User has a MATLAB function and wants faster GPU MEX or CUDA code
• User mentions GPU Coder, codegen, CUDA, gpuPerformanceAnalyzer
• User asks to profile generated GPU/CUDA code or GPU MEX (profiling generated

GPU code is the entry point to this skill's diagnostic-fix workflow)

• User wants to reduce GPU memory, improve kernel parallelism, or fix Performance Analyzer diagnostics
• User has a .m design file and representative inputs

When NOT to Use

• Workflows with no codegen
• coder.gpuConfig("exe") — standalone executables cannot be benchmarked or

equivalence-checked from MATLAB. Suggest the user switch to mex or lib/dll and regenerate exe from the final optimized source.

• Simulink GPU code generation
• Writing new MATLAB functions from scratch (this skill optimizes existing code)
• Optimizing helper functions called by the design file — this skill optimizes the main design file only
• Hardware setup or CUDA toolkit installation
• General MATLAB performance tuning without GPU involvement
• The user's prompt does not mention GPU, codegen, CUDA, MEX, or profiling.

Activation must be driven by the user's prompt alone — do not infer GPU intent from filenames or function contents. If unsure, ask before activating.

Workflow

Setup — Create Session Directory

All codegen artifacts, profiling outputs, and optimized versions go in a single temp directory for the entire session. pwd must be sessionDir for every codegen/benchmark/PA call — otherwise compiled MEX and SIL binaries land in the user's working directory.

Two helpers this skill calls — benchmarkMex and extractDiagnostics — live in the scripts/ subfolder of this skill (the folder containing this SKILL.md). A MATLAB function is only callable by name when its folder is on the path, so add scripts/ to the path in Setup and remove it in the cleanup. Then call the helpers by bare name (benchmarkMex(...), extractDiagnostics(...)).

% TEMPLATE — not executable
sessionDir = fullfile(tempdir, "gpu_opt_" + string(datetime("now", Format="yyyyMMdd_HHmmss")));
mkdir(sessionDir);
scriptsDir = fullfile("<skill_dir>", "scripts");   % this skill's scripts/ folder (absolute)
addedDir = fileparts(which("<designFile>"));
addpath(scriptsDir, addedDir);
oldDir = cd(sessionDir);
cleanupCd = onCleanup(@() (cd(oldDir), rmpath(scriptsDir), rmpath(addedDir)));   %#ok<NASGU>

Write all artifacts to sessionDir only — never to the user's working directory.

Step 1 — Codegen on Original

Run codegen on the unmodified design file to discover what actually fails. Do NOT guess which functions are unsupported — let the compiler tell you.

1a. Pick the codegen config. Use the config the user provides. If none specified, default to MEX:

cfg = coder.gpuConfig("mex");  % default — replace if user specifies a config

Supported targets: mex, lib, dll. The exe target is not supported by this skill, so Steps 2–5 cannot benchmark or verify equivalence. If the user provides coder.gpuConfig("exe"), stop and ask them to either:

• switch to mex for the optimization workflow (recommended — fastest

iteration), or

• switch to lib/dll if they need a deployable artifact (the skill will

enable SIL to benchmark via a generated MEX).

Once optimization is complete, the user can regenerate with exe from the final optimized source.

1b. For lib/dll configs: enable SIL. This is mandatory — without SIL there is no callable MEX, so Steps 2–5 cannot benchmark. Set this before calling codegen:

% TEMPLATE — not executable
if ~isa(cfg, 'coder.MexCodeConfig')
    cfg.VerificationMode = 'SIL';   % required for benchmarking lib/dll targets
end

If SIL fails or is unavailable (e.g., Embedded Coder license missing), report this and stop — do not silently skip benchmarking.

1c. Resolve inputs. If the user provided concrete input values, use them as-is. If the user provided only types/sizes (e.g., "two double vectors of size 1024x1"), synthesize inputs matching the spec — record exactly what you generated (type, size, location, generator) so the Final Report can list it. A reasonable default is randn with a fixed rng seed for floats, randi for integers, rand > 0.5 for logicals; keep inputs on the CPU unless the user said otherwise or PA later flags UseGpuInput. If the user gave neither values nor types/sizes, ask for representative sizes and types — codegen -args needs a concrete signature and the input shape drives which optimizations win.

1d. Run codegen:

% TEMPLATE — not executable
codegen -config cfg <designFile> -args {<inputs>}

If codegen fails, read the errors and fix only what is reported as unsupported. Save the fixed file as <designFile>_v1.m in sessionDir.

v1 must successfully codegen. Re-run codegen until it passes. This produces the baseline MEX: <designFile>_v1_mex (for lib/dll configs, the SIL-generated MEX has the same name and is callable identically).

Step 2 — Baseline Benchmark

Use the MEX generated in Step 1 directly — do not re-codegen. Benchmark with the convergence-based helper:

% TEMPLATE — not executable
baseline = benchmarkMex("<designFile>_v1_mex", {<inputs>});
baselineTime = baseline.MedianTime;
fprintf("Baseline: %.4f ms\n", baselineTime*1000);

Rules:

• Always use benchmarkMex (or gputimeit) — never use tic/toc for GPU timing (GPU ops are async)
• benchmarkMex handles warmup and convergence automatically
• Record baselineTime — all improvements are measured against this

Step 3 — Structural Optimization Loop

Apply optimizations iteratively. Each iteration:

1. Create <designFile>_v<N>.m in sessionDir with the next optimization
2. Verify numerical equivalence against the original (multi-input — see Step 3b)
3. Run codegen to sessionDir — if it fails, fix or revert
4. Benchmark the new MEX with benchmarkMex — compare against best so far
5. Keep the fastest passing version as the current best

MATLAB-level restructuring patterns (rewrites that improve codegen regardless of which GPU primitive you eventually choose):

The table above covers MATLAB-level restructuring only; it does not enumerate GPU Coder primitives. Before committing to any optimization, read references/gpu-codegen-functions.md end-to-end — it documents kernel pragmas, parallel reductions and scans, atomics, stencils, and memory placement. The right primitive depends on the loop's data-flow shape (where the result lives, how dependencies chain, whether iterations collide); the closest-looking table row is often not the right answer. Match semantics to the loop, not surface appearance.

Exit criteria for this loop:

• Performance target met (if user specified one)
• No more structural optimizations the agent can identify
• Maximum 5 structural iterations (Step 5's diagnostic-fix loop has its own

separate cap of 5 — the two are independent)

Step 3b — Verify Numerical Equivalence

Run both original and optimized on up to 5 input sets. Use the user's original inputs as the baseline, then generate variants matching the same types and sizes. Match the generator to the input type (randn for float, randi for integer, rand > 0.5 for logical) and keep values in the type's valid range.

% TEMPLATE — not executable — float case; adapt generator to the input type
% Input sets — variants are built from in1, the first input resolved in Step 1c
in1 = <user_or_synthesized_input_from_step_1c>;     % original example
rng(42); in3 = randn(size(in1), 'like', in1);       % random
rng(99); in4 = randn(size(in1), 'like', in1) * 1e6; % large magnitude

Include an edge-case input only where it is valid for this function — an all-zeros set exercises little and is degenerate if the function divides by something derived from the input. Pick a variant that meaningfully stresses the function instead.

Compare each output independently:

% TEMPLATE — not executable
outOrig = <designFile>(testInput);
outOpt  = <designFile>_v<N>(testInput);
 
% Gather gpuArray outputs to CPU for comparison
if isa(outOpt, 'gpuArray'), outOpt = gather(outOpt); end
 
% If outputs can contain NaN, first check NaN positions match (see NaN rule
% below) — max() ignores NaN, so a corrupted NaN would otherwise pass silently.
denom = max(abs(outOrig(:)));
if denom == 0
    relErr = max(abs(outOpt(:)));  % absolute error when expected is zero
else
    relErr = max(abs(outOrig(:) - outOpt(:))) / denom;
end
fprintf("relErr = %.2e\n", relErr);

Tolerance by output type:

Rules:

• Always compare against the original unmodified function, not the previous version
• If the function has multiple outputs, check each with its own type-appropriate tolerance
• If output types differ between original and optimized, that's a bug — fix it
• Normalize relative error by max(abs(expected)), not element-wise
• Handle NaN: check NaN positions match (isequal(isnan(a), isnan(b))), then compare non-NaN elements
• gpuArray wrapping of inputs is allowed (e.g., after UseGpuInput fix) — it doesn't change the function contract

Step 4 — Profile with gpuPerformanceAnalyzer

Always run this step unless the user specified a performance target and it was met in Step 3. PA reveals issues (memory copies, low parallelism) that benchmarks alone cannot detect.

Run PA on the current best version to find remaining bottlenecks. Pass the same cfg from Step 1 so PA profiles the same target the user is optimizing for:

% TEMPLATE — not executable
paDir = fullfile(sessionDir, "pa_v<N>");
mkdir(paDir);
 
gpuPerformanceAnalyzer("<designFile>_v<N>", {<inputs>}, ...
    Config=cfg, ...
    OutFolder=paDir, ...
    LaunchReport=false);

Key NVPs:

• Config=cfg — reuse the codegen config from Step 1 (user-provided or

the default coder.gpuConfig("mex"))

• LaunchReport=false — suppresses GUI report
• OutFolder — subdirectory within session dir

Extract diagnostics from the mldatx report:

% TEMPLATE — not executable
mldatxFile = fullfile(paDir, "html", "gpuProfiler.mldatx");
results = extractDiagnostics(mldatxFile);