aws-neuron/neuron-agentic-development/neuron-explorer-profile-schema

Neuron Explorer Profile Schema

The neuron-explorer profile schema is the OpenAPI-typed contract for the parquet tables produced when a profile is ingested. This skill is a starting point — the schema evolves with each neuron-explorer release, so always fetch the version that matches your installed CLI before relying on field names or types.

When to use this skill

• You need to know what tables exist and what each one represents.
• You need to look up fields, units, types, or enum values for a specific

table.

• You need to find which fields relate to a topic (matmuls, source lines,

HBM, semaphores, etc.).

• You want to understand where the values in a table come from (which input

artifact, which producer in the Neuron stack).

For running queries against an ingested profile, use /neuron-nki-profile-querying. For capturing a profile, use /neuron-nki-profiling.

Profile output format guidance: prefer Parquet over JSON

When using neuron-explorer view avoid using the legacy --output-format json because it is disorganized, incomplete, poorly documented, and not scalable. The default output format is Parquet. Parquet is a columnar and efficiently encoded format, so producing/manipulating/analyzing Parquet is fast even for large profiles. The neuron-explorer UI reads from the same Parquet, so what you query matches what the UI shows.

Also, note that --output-format summary-json maps to the data in the Summary.parquet table — same fields, same values. Prefer the default, full Parquet output over summary-json because it takes about the same time to produce and contains more information.

Get the schema for your installed neuron-explorer

The schema is embedded in the neuron-explorer binary. To dump it:

neuron-explorer --show-profile-schema | \
    scripts/write_profile_schema_to_separate_yaml_files.py --out ./schema

--show-profile-schema prints all per-table YAML files concatenated with --- separators. The bundled helper script splits the stream into one file per table, naming each by the top-level YAML key (Instruction.yaml, Summary.yaml, etc.). The OpenAPI root document is written to schema.yaml.

Pitfall: an outdated local copy of the schema causes confusing analysis errors — fields renamed/removed, new columns missing, enums out of sync. Re-fetch the schema any time you upgrade neuron-explorer or switch between hosts with different versions installed.

Quickly scan the schema

You can string search across the entire profile schema by searching in the YAML files dumped above (no profile needed):

# List every table and its description
grep -A1 -E '^[A-Z][A-Za-z]+:$' ./schema/*.yaml | grep -E '^(./schema|--|.*description:)' | head -40
 
# All fields whose name or description mentions "matmul" or "MATMUL"
grep -inE 'matmul' ./schema/*.yaml
 
# All fields that link to source code (file/line/function/stack)
grep -inE 'stack_frame|source_location|source_line|file_path|line_number|function_name|kernel_file|kernel_line' ./schema/*.yaml

Alternatively, the same schema information can be obtained in the SchemaFields table. After processing a profile the SchemaFields.parquet table will exist in the profile output. This table is self-describes the entire profile schema (same as the YAML, but as a queryable table). Each row describes a (table, field) with type, description, unit, format, requiredness, min/max, and enum values. You can either query the SchemaFields table using DuckDB SQL or via the API server after ingesting any profile (more details on both methods are documented in the /neuron-nki-profile-querying skill):

curl -s -X POST http://localhost:3002/api/v1/db/${PROFILE_NAME}/_search \
    -H 'Content-Type: application/json' \
    -d '{"type":"databaseExplorerQuery","tableName":"SchemaFields",
         "query":"SELECT table_name, field_name, field_description, field_unit
                  FROM SchemaFields WHERE field_name ILIKE '\''%matmul%'\''"}' \
    | python3 -m json.tool

Modalities — kinds of data in the profile

Every table is one of the following modalities. Knowing the modality tells you how to read the rows.

Data flow at a glance

Knowing where data originates helps when the schema description doesn't fully answer a question — you can chase the field back to the producer's source code.

flowchart LR
    HW["Neuron Hardware<br/>(NeuronCore notifications,<br/>throttle, errors)"]
    NRT["Neuron Runtime<br/>(libnrt, nrt_inspect)"]
    FW["Frameworks<br/>(torch.profiler,<br/>jax.profiler, vLLM)"]
    CC["Neuron Compiler<br/>(neuronxcc)"]
    NKI["NKI front-end<br/>(lowering pass)"]
    USR["User<br/>(upload --source-code<br/>or UI dialog)"]
 
    NTFF["*.ntff<br/>(binary device trace)"]
    NTRACE["ntrace.pb +<br/>cpu_util.pb +<br/>host_mem.pb +<br/>trace_info.pb"]
    CTJ["*/plugins/*/<br/>trace.json.gz<br/>(Chrome Trace JSON)"]
    NEFF["*.neff archive<br/>+ debug_info/*.dbg"]
    KDI["kernel_debug_info.json<br/>(in NEFF)"]
    SRC["source_folder.tar.gz<br/>(side-car archive)"]
 
    NE{{"neuron-explorer<br/>ingest + derive"}}
 
    OUT["parquet tables"]
    UI["UI source rendering<br/>(served by /fs/* API,<br/>not a table)"]
 
    HW --> NTFF
    NRT --> NTFF
    NRT --> NTRACE
    FW --> CTJ
    CC --> NEFF
    NKI --> KDI
    CC --> KDI
    USR --> SRC
 
    NTFF --> NE
    NTRACE --> NE
    CTJ --> NE
    NEFF --> NE
    KDI --> NE
    SRC --> NE
 
    NE --> OUT
    NE --> UI

Producer → input artifact → neuron-explorer -> output data table:

Source code linking

A profile is most useful when trace data can be traced back to the user code that produced it. The schema captures four distinct kinds of "user code" attribution. Knowing which one(s) your profile has tells you what attribution queries will actually return data.

1. Framework op hierarchy string (compiled flows)

Where the op sits in the model: a slash-separated path of nn.Module names from the model root down to the op (e.g. GPT2Model[model]/Attention[attn]/Linear[q_proj][2]/aten.linear). Captured at compile time by the framework's tracer; populated only for compiled PyTorch flows.

2. Python source location + stack frame index (compiled flows)

The Python file, line number, and function name for each instruction, plus the full caller chain. Captured from PyTorch frame info at compile time and embedded in the NEFF debug info; empty for eager-mode profiles.

A single instruction may carry multiple stack_frame_ids because compiler fusions collapse multiple source locations onto one hardware instruction.

3. NKI kernel debug info

For NEFFs containing NKI kernels: the kernel source file and line number for each kernel instruction, plus the kernel call stack and the surrounding loop-nest iteration variables. Captured by the NKI front-end and bundled into the NEFF; empty for non-NKI workloads.