zhongjingyun/codex-plugins/physical-ai-neural-reconstruction

Physical AI Neural Reconstruction (NuRec) Router

Purpose

This is a thin router for NVIDIA Neural Reconstruction (NuRec) requests. It points at the upstream nurec-index skill at https://github.com/NVIDIA/nurec-skills and its five sibling skills (physical-ai-datasets, ncore, nre, asset-harvester, nurec-fixer). Use this skill to:

• Identify which upstream sibling skill answers a NuRec question.
• Locate, clone, or refresh the canonical nurec-skills checkout.
• Order multi-step NuRec workflows (data → conversion → train →

render → cleanup) before opening the upstream recipe.

The canonical recipes (training, rendering, data conversion, dataset downloads, object harvesting, frame cleanup) live in the upstream sibling skills. Never copy or reconstruct their commands here.

Do NOT use this skill for:

• SimReady packaging of CAD or source meshes → use

omniverse-cad-to-simready.

• Generic USD performance tuning unrelated to NuRec → use

omniverse-usd-performance-tuning.

• AKS / OSMO / NIM Operator infrastructure setup → use

physical-ai-infrastructure-setup-and-resilient-scaling.

When to Use

Read this skill first whenever a user mentions any of:

nurec, nurec router, nurec index, neural reconstruction, neural reconstruction engine, NRE, 3DGUT, 3DGRT, USDZ, NCore V4, sensor sim, novel view synthesis, PhysicalAI-Autonomous-Vehicles-NuRec, PhysicalAI-NuRec-PPISP, Cosmos-Drive-Dreams, asset harvester, nurec fixer, DiffusionHarmonizer, harmonizer, difix, difix3d, serve-grpc, render-grpc, warm serve-grpc, nre thin client, batch_render_rgb, nurec teardown, "where do I start with NuRec", "which NuRec skill should I use for X?".

Decide which upstream sibling skill answers the question, fetch it (see Locate and fetch the upstream skills), then follow that skill's body.

Prerequisites

Router skill itself has no runtime prerequisites beyond git for fetching the upstream. Downstream sibling skills require:

• Docker + NVIDIA Container Toolkit + GPU — for nre, nre-tools,

and nurec-fixer containers (nvcr.io/nvidia/nre/nre, nvcr.io/nvidia/nre/nre-tools, nvcr.io/nvidia/cosmos/cosmos-predict2-container:1.2).

• NGC API key (NGC_API_KEY) — for pulling NGC containers.
• Hugging Face token (HF_TOKEN) with the

nvidia/PhysicalAI-*, nvidia/DiffusionHarmonizer, and nvidia/asset-harvester gated licenses accepted in advance on Hugging Face.

• Python 3.10+ with huggingface_hub installed.
• (Optional) CARLA, Isaac Sim 5.1, or AlpaSim for simulator

integration over serve-grpc.

Verify secrets safely (do not echo values):

hf auth whoami
[ -n "${HF_TOKEN:-}" ]      && echo "HF_TOKEN length=${#HF_TOKEN}"      || echo "HF_TOKEN unset"
[ -n "${NGC_API_KEY:-}" ]   && echo "NGC_API_KEY length=${#NGC_API_KEY}" || echo "NGC_API_KEY unset"

See `references/secrets-handling.md` for the bash anti-patterns to avoid.

What is NuRec?

NuRec (NVIDIA Omniverse Neural Reconstruction) takes camera, LiDAR, radar, or stereo recordings — typically from a self-driving car or a robot — and turns them into a 3D scene you can re-render from any viewpoint. Names that come up a lot:

• NRE — "Neural Reconstruction Engine". NuRec is the product; NRE

is the engine that trains and renders. Both route to the upstream nre skill.

• USDZ — the file format of a trained scene. A zip archive that

Omniverse, Isaac Sim, and CARLA can open.

• NCore V4 — the input format NRE consumes. Raw recordings must be

converted to NCore V4 before training.

• 3DGUT / 3DGRT — the two 3D Gaussian Splatting flavours used

internally by NRE. The default Hydra recipe picks one; most users never set it manually.

A typical NuRec project has three stages:

1. Get the input — convert your own recording to NCore V4

(ncore), or download a pre-converted dataset (physical-ai-datasets).

1. Train the reconstruction — feed NCore V4 to NRE; out comes a

USDZ (nre).

1. Render new views — render images, videos, or LiDAR sweeps from

the USDZ (nre).

Projects that just want to use an existing NVIDIA-published scene skip step 2.

Pick a skill

Match the user's goal in the left column and open the named upstream skill on the right. Arrows mean "do these in order".

Common workflows

Six end-to-end workflows are documented in `references/workflows.md`:

• A. Make a NuRec scene from your own recording.
• B. Use a NuRec scene NVIDIA has already trained.
• C. Add, remove, or replace 3D objects in a scene.
• D. Clean up rendered frames.
• E. Benchmark reconstruction quality.
• F. Connect NuRec to a simulator.

Open that file when the user's task spans more than one sibling skill.

Sibling skills (upstream)

For naming overlaps (NRE vs Fixer, ncore vs nre, AV-NuRec vs Cosmos-Drive-Dreams, NuRec vs SimReady) see `references/mix-ups.md`.

Locate and fetch the upstream skills

Quick recipe (full version in `references/upstream-fetch.md`):

UPSTREAM_ROOT="${NUREC_SKILLS_UPSTREAM_ROOT:-${PHYSICAL_AI_SKILL_HUB_UPSTREAM_ROOT:-$HOME/.physical-ai-skill-hub/upstreams}}"
mkdir -p "$UPSTREAM_ROOT"
if [ -d "$UPSTREAM_ROOT/nurec-skills/.git" ]; then
  git -C "$UPSTREAM_ROOT/nurec-skills" fetch --tags
  git -C "$UPSTREAM_ROOT/nurec-skills" checkout main
  git -C "$UPSTREAM_ROOT/nurec-skills" pull --ff-only
else
  git clone --depth 1 https://github.com/NVIDIA/nurec-skills.git \
    "$UPSTREAM_ROOT/nurec-skills"
fi
test -f "$UPSTREAM_ROOT/nurec-skills/.agents/skills/SKILL.md"

Then read the upstream skill before running any mutating command:

cat "$UPSTREAM_ROOT/nurec-skills/.agents/skills/SKILL.md"          # router
cat "$UPSTREAM_ROOT/nurec-skills/.agents/skills/<folder>/SKILL.md" # sibling

Local lookup order (try in order before the upstream clone):

1. .agents/skills/<name>/SKILL.md (Cursor, Codex, NemoClaw)
2. .claude/skills/<name>/SKILL.md (Claude Code)
3. .cursor/skills/<name>/SKILL.md (project-scoped)
4. ~/.cursor/skills/<name>/SKILL.md (personal skills)

Hard Rules

• Router only — do not duplicate upstream NuRec recipes here. Read

the upstream sibling skill body before running any mutating command.

• Refer to sibling skills by their name: (e.g. nre), not by repo

path. Folder layouts can change; the name is portable.

• Clone or refresh https://github.com/NVIDIA/nurec-skills under the

shared upstream root (${NUREC_SKILLS_UPSTREAM_ROOT:-${PHYSICAL_AI_SKILL_HUB_UPSTREAM_ROOT:-$HOME/.physical-ai-skill-hub/upstreams}}/nurec-skills). Do not scan broad developer workspaces such as ~/Codes or reuse unrelated old clones.

• physical-ai-datasets covers gated Hugging Face datasets. Do not

bypass dataset license terms; the user must accept the PhysicalAI-* gated licenses on Hugging Face and provide a token before downloading.

• Asset Harvester runs before packaging into a USDZ. Do not call

nre's export-external-assets on hand-rolled .ply files unless the user explicitly asks to skip Asset Harvester.

• For artifact cleanup, prefer the built-in --enable-difix path in

nre. Route to the standalone nurec-fixer only when the user needs the public code/model card, paired evaluation, fine-tuning, or fixes on previously rendered frames.

• Do not invent NRE / NCore / DiffusionHarmonizer commands from

memory. Re-read the upstream sibling skill — versions move fast (NRE release_26.04 is the current pinned tag).

• This router does not deploy infrastructure. Route AKS / OSMO /

NIM Operator setup to physical-ai-infrastructure-setup-and-resilient-scaling.

Limitations