nvidia-omniverse/ovrtx/spg

Sensor Processing Graphs

When to Use

Any task where the user's own GPU code has to run inside the renderer, over the images or sensor data the renderer produces. Trigger vocabulary: SPG, .cu.lua, .slang.lua, info:spg:sourceAsset, subIdentifier, omni:rtx:aov, orderedVars, cuda.kernel, slang.compute, slang.dispatch, spg:rtx.spg.stdlib/..., "post-process the AOV", "custom kernel on the render output", "my SPG AOV is black". Not this skill: reading a finished AOV back on the host, which is reading-render-output.

Task → Sections

Read The Model, Instructions and The Traps for any task. Then only what the row names.

Prerequisites

• ovrtx 0.4 or later. SPG is enabled by default.
• A working stage, camera RenderProduct and step loop; Slang requires Vulkan (the default).
• CUDA or Slang/HLSL compute and USD authoring are assumed; this skill maps them onto SPG.

Inputs

The user's scene file, their kernel if they have one, and which backend they want. If they have not said, choose by what the node needs: Choosing a Backend.

Every `examples/` and `docs/` path below is relative to the ovrtx public root: rendering/ovrtx/public/ in the kit repository, the package root in an installed ovrtx.

Precedence when facts conflict: the code under examples/python/spg-* first, then the pages under docs/spg/, then this skill. Every code reference below names a snippet marker in a runnable example, a pair of comments // [snippet:name] and // [/snippet:name] (-- in .lua, # in .py and .usda); read the file between the markers, not memory.

The Model

SPG runs a directed graph of GPU nodes inside the renderer's frame, over its Arbitrary Output Variables (AOVs). Everything stays on the GPU.

A node is three files that share a name: the GPU code, Kernel.cu or Kernel.slang; the launch script beside it, Kernel.cu.lua or Kernel.slang.lua, which validates inputs, describes outputs and returns the launch configuration; and the USD shader definition, Kernel.usda, which declares the ports and points at the GPU source.

Three facts drive most mistakes:

1. The launch script is found by appending `.lua` to the source asset path, and cannot be

redirected. GrayscaleKernel.cu pairs with GrayscaleKernel.cu.lua.

1. The launch script's function runs once per rendered frame, on the CPU, before the GPU

work, and sees resource descriptors (shape, dtype, rank), never pixel or point data.

1. Execution is driven backwards from what is asked for. A node whose output nothing

consumes and which publishes no RenderVar never runs at all.

Instructions

1. Decide built-in or custom. If a stdlib node covers the operation, author only a USD

Shader prim and write no code: Built-In Nodes. Do not invent node IDs.

1. Write the GPU code, one entry point, named the same in all three files and extern "C"

on CUDA so the symbol survives.

1. Write the launch script, one global Lua function of that name taking

(inputs, outputs), assigning a descriptor to every declared output, which is otherwise never backed, and returning a launch configuration.

1. Write the shader definition: info:implementationSource = "sourceAsset",

info:spg:sourceAsset at the GPU file, subIdentifier naming the entry point, and one opaque inputs:/outputs: port per resource. Details in The Three Files.

1. Wire it into the scene: Wiring the Graph.
2. Verify by content, not by exit code. A node that never ran leaves an AOV of zeros, so a

clean run proves nothing. Assert something only your node could have produced, such as R == G == B in every pixel of a grayscale output. If you cannot run anything, hand the user the command and the check rather than reporting the work as verified.

The smallest complete node, in all three files:

Source (kernel): examples/python/spg-grayscale/GrayscaleKernel.cu snippet grayscale-kernel-template

Source (launch script): examples/python/spg-grayscale/GrayscaleKernel.cu.lua snippet grayscale-launch-template

Source (shader definition): examples/python/spg-grayscale/GrayscaleKernel.usda snippet shader-definition-template

Source (scene): examples/python/spg-grayscale/grayscale_scene.usda snippet render-graph

The Three Files

The entry-point name appears three times and must match exactly: the GPU source's function, the launch script's global function, and subIdentifier in USD, which falls back to the Shader prim's own name when absent.

Ports are declared in USD and keyed by name in Lua, with the inputs: scope stripped, so inputs:LdrColor arrives as inputs["LdrColor"]. A resource port is opaque inputs:X or opaque outputs:Y and carries an AOV or a buffer another node produced. A value input is a typed attribute, float inputs:strength, carrying a number, vector, matrix, token or asset.

A resource is described by `shape`, `rank` and `dtype`. A dtype is a value taken from the cuda or slang table: cuda.uchar4 is four unsigned 8-bit numbers, so one RGBA pixel. It compares, inputs["Image"].dtype == cuda.uchar4, and it is callable to wrap a value, cuda.int(42).

Shapes are height-first. shape[1] is height and shape[2] is width, while cuda.image(width, height, dtype) takes width first. That reversal is the most common cause of a transposed or wrongly proportioned result.

Choosing a Backend

CUDA and Slang are the same shape: same three files, same ports, same launch-script contract. They differ in how the GPU code receives things.

The same node in both, to compare directly:

Source (CUDA): examples/python/spg-pipeline/InvertKernel.cu.lua snippet invert-launch

Source (Slang): examples/python/spg-pipeline/InvertKernel.slang.lua snippet invert-slang-launch

Values

Declare a typed attribute on the shader definition; a scene instance may override it.

Source (USD): examples/python/spg-pipeline/InvertKernel.usda snippet invert-shader-definition

A value input is a wrapper, not a number. Arithmetic on inputs["strength"] raises. Pass it through a dtype constructor to hand it to the GPU, or read .value to compute with it in Lua.

Slang integer fields require matching element widths. Narrower integers are rejected with an error naming the value and widths. Use slang.int or slang.uint for a 32-bit field.

A numeric vector can pass by value on CUDA. A dtype constructor passes every component to a kernel parameter of the matching type and component count. cuda.array passes a pointer to the components instead; use it for matrices or when the kernel expects a pointer. A quaternion can use a matching four-component dtype constructor or cuda.array. On Slang every value goes into the parameter block.

An array may hold no elements. Its sourceShape[1] is 0, cuda.array gives a null pointer and a Slang buffer a placeholder, so pass the count to the GPU code and read no element past it.

Traps in the type mapping. A quaternion is written (w, x, y, z) in USDA and arrives as (x, y, z, w). A token arrives as a null-terminated char array and an asset as the file's raw bytes. CUDA passes a token with array. A Slang uint4 field needs token bytes packed explicitly into four 32-bit words; see docs/spg/do/values.rst. Assets use array with the dtype the shader reads. A token[] input is not taken: it keeps the node's graph from being built, with an error naming the node and the input.

Depth, including every USD type SPG reads: docs/spg/ref/usd.rst and docs/spg/do/values.rst.

Resources

Backing is fixed when a resource is created. image asks for a texture, empty for a buffer, and the binder used later has to agree. A texture has one, two or three dimensions; anything else is buffer-backed.

On Slang the binder must match how the shader declares the parameter. The binders check rank, direction, backing and buffer kind in Lua before anything reaches the GPU, naming the port.

A `uchar4` texture is normalised, alone among the 8-bit element types, so Slang writes float4 in 0..1 with hardware rounding while CUDA writes raw bytes. The same colour can land one least-significant bit apart on the two backends; where a test must match byte for byte, pick colours exact in 8 bits.

Texture in and texture out, then buffers and textures mixed in one bind list:

Source (CUDA): examples/python/spg-grayscale/GrayscaleKernel.cu.lua snippet grayscale-launch-template

Source (Slang): examples/python/spg-grayscale/GrayscaleKernel.slang.lua snippet grayscale-slang-launch-template

Source (mixed): examples/python/spg-composite-aov/RangeHistogramKernel.cu.lua snippet histogram-binding

Depth: docs/spg/do/textures_buffers.rst.

Launch Geometry

`block` and `grid` are optional; SPG derives them. Left out, the geometry comes from the first output's shape: one thread per element. On CUDA the derived block is 16 x 16 x 1, or 256 x 1 x 1 for a rank-1 or rank-0 resource.

Without a block override, any blockDimX, blockDimY or blockDimZ value input makes the missing axes default to 1. An explicit launch block takes precedence over those inputs.

On Slang, `numthreads` states the group size a `.spv` was compiled with, which it does not reflect; nothing checks it against the byte code, so a wrong value mis-covers the output. A shader that declares [numthreads] supersedes it, logged only at INFO level.

State the geometry when the iteration domain is not the output. A kernel whose threads walk a buffer, or take one thread per column rather than per pixel, must say so. Round up when dividing, or the last partial group never launches and the far edge is never written. Both cases, one pass stating nothing and one stating the domain:

Source (derived): examples/python/spg-blur/BlurKernel.cu.lua snippet blur-horizontal-launch

Source (stated): examples/python/spg-blur/BlurKernel.cu.lua snippet blur-vertical-launch

Depth: docs/spg/do/launch_geometry.rst.

Wiring the Graph

A graph is authored under a RenderProduct: a RenderVar names an AOV, orderedVars lists what the product produces, and a shader-to-shader connection chains two nodes without publishing what passes between them.