openvinotoolkit/openvino/add-fusion-transformation

Agent Skill: Add New Fusion Transformation in OpenVINO

Goal

Provide an instruction-only workflow to implement a new OpenVINO graph fusion transformation that replaces a matched subgraph with one operation (or a compact operation sequence), including regression tests.

Expected outcome:

• New transformation pass implementation
• Pass registration in the appropriate pipeline
• Positive and negative transformation tests
• Functional coverage when behavior is user-visible

Scope

This workflow applies to OpenVINO transformation development in openvino/ for:

• Fusion transformations — pattern-based graph fusion that replaces a matched subgraph with:
• a single fused op, or
• several ops preserving semantics with lower runtime overhead
• _Purpose: optimization (reduced dispatch overhead, better hardware mapping)_

• Decomposition transformations — pattern-based replacement that expands a complex op into a

sequence of simpler, already-supported ops:

• _Purpose: enablement prerequisite — makes an unsupported op runnable on all backends

without adding a new core op. Use when the semantics can be faithfully expressed via existing ops._

• Decomposition is the first thing to evaluate when a new op is needed. If the op is

decomposable, no new core op class is necessary.

The workflow covers:

1. Pattern design and safety guards
2. Pass implementation and registration
3. Runtime correctness constraints (shape/type/friendly name/rt_info)
4. Unit and functional regression tests

Architecture

Model Graph → Pattern Matcher Pass → Subgraph Replacement → Validation/Inference Tests

Expected Repository Layout

openvino/ ├─ src/common/transformations/ │ ├─ include/transformations/ │ │ └─ <domain>/<fusion_name>.hpp │ ├─ src/transformations/ │ │ └─ <domain>/<fusion_name>.cpp │ ├─ src/transformations/common_optimizations/ │ │ └─ common_optimizations.cpp (or relevant pipeline file) │ └─ tests/ │ ├─ common_optimizations/ │ │ └─ <fusion_name>.cpp │ └─ utils/ └─ tests/ └─ layer_tests/ or functional/ (if user-visible behavior/perf contract is affected)

Instruction Workflow

1) Confirm the fusion does not already exist

• Search for equivalent matcher patterns and passes.
• If similar fusion exists, extend it instead of creating a duplicate pass.
• Reuse existing helper utilities and pattern predicates where possible.

2) Define fusion contract before coding

• Specify exact matched subgraph topology and constraints.
• Specify replacement graph (single op or multi-op compact form).
• Document semantic invariants:
• output values
• element types
• ranks/dynamic-shape compatibility
• runtime info / friendly names
• Identify explicit no-fuse conditions (negative guards).

3) Implement matcher pass

• Add pass class declaration in header and implementation in source.
• Prefer ov::pass::MatcherPass for local rewrite; use GraphRewrite only when grouping multiple related matchers.
• Make predicates strict enough to avoid unsafe rewrites.
• Preserve node names and rt_info where expected.
• Register and copy runtime info for replacement nodes.
• Avoid hidden behavior changes; if assumptions are violated, do not fuse.

4) Register pass in the right optimization pipeline

• Add the pass to the appropriate transformation pipeline file.
• Keep registration order consistent with dependencies (producer fusions before consumer fusions when required).
• Do not widen pass scope beyond the intended domain.

5) Add transformation unit tests

• Add tests under transformations test suite with:
• positive case(s): pattern is fused
• negative case(s): pattern must remain unchanged when guards fail
• dynamic-shape/type coverage where relevant
• Compare transformed function against expected reference graph.
• Ensure tests validate both structure and semantics.

6) Add functional/layer tests when externally observable

• If fusion changes user-visible behavior/performance-sensitive execution path, add/extend functional tests.
• Cover representative input shapes/dtypes and backend-relevant conditions.
• Keep tests minimal and deterministic.

7) Validate and finalize

• Ensure pass is deterministic and idempotent.
• Verify no duplicated matcher registration.
• Confirm no fallback behavior silently changes model semantics.
• Report one of:
• fully implemented + covered,
• partially implemented (missing functional coverage),
• blocked with explicit reason.

Fusion Design Recommendations

• Prefer root-cause optimization: fuse only when the resulting graph is strictly safer/faster or equivalent.
• Favor existing OpenVINO ops over introducing new custom ops unless required.
• Keep pattern predicates robust for partial/dynamic shapes.
• Avoid expensive checks in hot transformation loops when cheaper predicates exist.
• Reuse canonical helper utilities for constants, shape checks, and node replacement.

Notes

• Keep this skill instruction-only in markdown.
• Do not claim fusion support without both positive and negative tests.
• Avoid broad refactoring unrelated to the fusion objective.
• Preserve compatibility with existing passes and test baselines.