kaichen-z/stream3d/sam3d-gso-manual-registration

Name: sam3d-gso-manual-registration
Author: kaichen-z

Claude Code and agent skill by kaichen-z/stream3d

Trust tier 2

community

LLM review pending

By kaichen-z
AI/ML
kaichen-z/stream3d
docs/skills/sam3d-gso-manual-registration/SKILL.md
Updated 2 Nov 2026
Version 1.0.01 version recorded

Source check pending

Checking that the SKILL.md file is still in the repository.

Use when a Streaming SAM3D GSO result needs a stable manual or agent-in-the-loop global Sim(3) registration against raw GSO render_mvs_25 assets.

Install

npx vskill@latest install kaichen-z/stream3d/sam3d-gso-manual-registration

SKILL.md

SAM3D GSO Manual Registration

This skill provides a direct source-geometry global Sim(3) workflow for GSO30 cases where the result_pose.npz camera chain is not trusted enough for evaluation. It is an auxiliary diagnostic loop, not a replacement for the formal automatic benchmark.

The script is:

bash

docs/skills/sam3d-gso-manual-registration/scripts/manual_register_sam3d_gso.py

The constrained Sim(3) adjustment helper is:

bash

docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py

The skill scripts are intentionally independent from other skill directories. They may use repository code under streaming/ and sam3d_objects/, but they must not import scripts from docs/skills/*.

Metric Reporting Rule

When reporting Chamfer Distance in paper tables, summaries, or user-facing comparisons, use cd_l2_mean by default. This is the unsquared symmetric nearest-neighbor L2 diagnostic and is the column used for comparing this skill's outputs with prior GSO manual-registration tables.

The final evaluator also writes cd_mean, which is PyTorch3D chamfer_distance(..., norm=2) on 4096 sampled pred/GT surface points. Treat cd_mean as squared-L2 Chamfer. Do not label it simply as CD in paper tables unless the table explicitly says squared Chamfer. If both are shown, use names such as:

text

CD-L2 = cd_l2_mean
CD-sq = cd_mean

The render-time summary.json field active_mesh_diagnostics.cd is also diagnostic-only and should not be used as the paper CD unless explicitly requested.

Inputs

Predicted chunk:

text

tmp/gso30-streaming-ablation/<variant>/<scene>/chunk_*/
  result.glb
  result.ply
  params.npz

Ground truth scene:

text

data/raw/gso/GSO30/<scene>/render_mvs_25/
  model_norm.glb
  model/{000..024}.png
  model/{000..024}.npy

result_pose.npz may exist and is recorded as provenance only. It is not used to estimate the global Sim(3).

When result.glb is missing, the official workflow allows params.npz to provide a sparse registration source:

load coords;
drop the batch column if present;
reorder latent [D,H,W] indices to local mesh-style XYZ with coords[:, [2, 0, 1]];
convert to canonical local coordinates with ((xyz + 0.5) / 64) - 0.5;
fit a 24-way signed-permutation centroid + uniform-scale + translation Sim(3) from params points to downsampled result.ply xyz;
use that locally aligned sparse cloud for global registration against render_mvs_25/model_norm.glb.

Final rendering still uses the chunk's result.ply Gaussian with the estimated global Sim(3). The sparse params.npz source is registration-only.

State File

Each run writes:

text

scratch/sam3d-gso-manual-registration/<variant>/<scene>/<chunk_name>/alignment_state.json

The stable fields are:

text

format_version
scene
chunk_root
gt_root
registration_space
source_mesh
source_geometry_kind
source_geometry_path
source_local_alignment
target_mesh
target_bbox_diagonal
needs_manual_review
auto_initial_sim3
active_sim3
registration_diagnostics
render_eval
artifacts

Both auto_initial_sim3 and active_sim3 contain:

text

scale
rotation
translation
matrix
source_space
target_space

active_sim3 is the only transform used by render. Agent or human edits should change active_sim3, leaving auto_initial_sim3 intact.

Workflow

First estimate the automatic initialization and render all 25 views:

bash

CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/manual_register_sam3d_gso.py \
  estimate \
  --scene alarm \
  --variant k2_4 \
  --chunk-root tmp/gso30-streaming-ablation/k2_4/alarm/chunk_0016 \
  --source-geometry auto

Inspect:

text

scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/
  comparisons/{000..024}_gt_pred.png
  comparison_contact_sheet.png
  per_view_metrics.csv
  summary.json
  alignment_state.json

For vision-model diagnosis, do not concatenate all 25 comparison images into one image. Review comparison pairs one at a time. The recommended first pass is 3 representative pairs, for example:

text

comparisons/000_gt_pred.png
comparisons/012_gt_pred.png
comparisons/024_gt_pred.png

Use the contact sheet only as a human index for choosing which individual pairs to inspect next.

If the visual overlay is not acceptable, use the constrained controller instead of raw 4x4 matrix editing. First create orientation candidates:

bash

./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  propose \
  --state-file scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/alignment_state.json \
  --round-name orientation_001 \
  --preset orientation

Render candidate comparison triples:

bash

CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  render-candidates \
  --round-dir scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/manual_adjustment_rounds/orientation_001 \
  --views 0,12,24

Inspect each candidate's individual comparisons/000_gt_pred.png, 012_gt_pred.png, and 024_gt_pred.png. Apply only a visually selected candidate:

bash

./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  apply-candidate \
  --state-file scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/alignment_state.json \
  --round-dir scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/manual_adjustment_rounds/orientation_001 \
  --candidate-id rot_z_180 \
  --reason "front/back orientation corrected in 000/012/024"

Then rerender the canonical state:

bash

CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/manual_register_sam3d_gso.py \
  render \
  --state-file scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/alignment_state.json

If orientation is now correct but a small residual remains, run a micro round:

bash

./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  propose \
  --state-file scratch/sam3d-gso-manual-registration/k2_4/alarm/chunk_0016/alignment_state.json \
  --round-name micro_001 \
  --preset micro \
  --rotation-step-deg 4 \
  --translation-step-fraction 0.01 \
  --scale-step 0.02

Repeat until the contact sheet and per-view comparisons are visually aligned. The final alignment_state.json is the reusable global Sim(3) record for that scene/chunk.

Configurable Initial GICP Variant

For scenes where TEASER++ chooses the wrong semantic orientation, use the pre-matched variant:

bash

CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/manual_register_sam3d_gso.py \
  estimate \
  --scene alarm \
  --variant k2_12 \
  --chunk-root tmp/gso30-streaming-ablation/k2_12/alarm/chunk_0016 \
  --registration-method initial_gicp_scale \
  --initial-count 60

This always evaluates the 24 proper signed-permutation rotations first. When --initial-count is larger than 24, it appends a deterministic Hopf-grid SO(3) rotation set until the requested count is reached. Typical values are 24, 36, 48, and 60. The legacy method name initial24_gicp_scale is still accepted; use --initial-count to choose the actual number of initial rotations.

Each initial is centroid/scale matched, refined with Open3D Generalized ICP for rotation/translation at fixed scale, then refit with a trimmed nearest-neighbor uniform scale/translation solve. The best candidate becomes both auto_initial_sim3 and active_sim3, and all candidate final_sim3 records are saved under registration_diagnostics.candidates.

For the ablation1 GSO past-solution outputs, most chunks contain params.npz and result.ply but no result.glb. Use:

bash

CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/manual_register_sam3d_gso.py \
  estimate \
  --scene alarm \
  --variant mvsam3d_flowedit \
  --chunk-root outputs/ablation1_gso_past_solution/mvsam3d_flowedit/alarm/chunk_0000 \
  --output-dir scratch/sam3d-gso-manual-registration-ablation1/mvsam3d_flowedit/alarm/chunk_0000 \
  --source-geometry auto \
  --registration-method initial_gicp_scale \
  --initial-count 60 \
  --views all

This automatically falls back from result.glb to params.npz for registration, while still rendering the transformed result.ply Gaussian.

When a scene from this variant is visually rejected, first export and render the saved registration candidates:

bash

./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  export-registration-candidates \
  --state-file scratch/sam3d-gso-manual-registration-initial-gicp/k2_12/alarm/chunk_0016/alignment_state.json \
  --round-name registration_candidates \
  --top-k 24
 
CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  render-candidates \
  --round-dir scratch/sam3d-gso-manual-registration-initial-gicp/k2_12/alarm/chunk_0016/manual_adjustment_rounds/registration_candidates \
  --views 0,12,24

Inspect candidate comparison triples one candidate at a time. Prefer a candidate that fixes semantic orientation even if its metrics are not rank 0. Apply the chosen candidate, then rerender the canonical state:

bash

./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/sim3_adjustment_controller.py \
  apply-candidate \
  --state-file scratch/sam3d-gso-manual-registration-initial-gicp/k2_12/alarm/chunk_0016/alignment_state.json \
  --round-dir scratch/sam3d-gso-manual-registration-initial-gicp/k2_12/alarm/chunk_0016/manual_adjustment_rounds/registration_candidates \
  --candidate-id rank_03_perm_xyz_sign_mmp \
  --reason "candidate fixes top/bottom orientation in individual 000/012/024 comparisons"
 
CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/manual_register_sam3d_gso.py \
  render \
  --state-file scratch/sam3d-gso-manual-registration-initial-gicp/k2_12/alarm/chunk_0016/alignment_state.json

Only after the best saved initial is chosen should subagents use propose --preset micro for small residual translation, rotation, or scale corrections.

Final Evaluation

After manual states and final 25-view renders are in outputs/sam3d-gso-manual-registration, run the evaluation stage:

bash

CUDA_VISIBLE_DEVICES=0 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/evaluate_manual_registration.py \
  --variants k2_4,k2_8,k2_12,k2_16 \
  --sample-points 4096 \
  --eval-dir outputs/sam3d-gso-manual-registration/evaluation

On an 8-GPU machine, it is fine to expose all cards:

bash

CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 ./.env/bin/python \
  docs/skills/sam3d-gso-manual-registration/scripts/evaluate_manual_registration.py \
  --variants k2_4,k2_8,k2_12,k2_16 \
  --sample-points 4096 \
  --eval-dir outputs/sam3d-gso-manual-registration/evaluation \
  --image-fid-backend timm_inception \
  --image-fid-batch-size 64 \
  --lpips-batch-size 16 \
  --point-feature-backend pointnet2_ssg \
  --pointnet2-repo scratch/external/Pointnet_Pointnet2_pytorch \
  --pointnet-checkpoint scratch/external/Pointnet_Pointnet2_pytorch/log/classification/pointnet2_ssg_wo_normals/checkpoints/best_model.pth \
  --point-feature-batch-size 8

Outputs:

text

outputs/sam3d-gso-manual-registration/evaluation/
  scene_metrics.csv
  scene_metrics.json
  metrics_summary.csv
  metrics_summary.json
  evaluation_run.json

Metric definitions:

This is the start of the file. Read the full SKILL.md on GitHub.

Security

Security scanPassed, 100 of 100Pattern checks on the skill's files.
Model reviewNot runA model review is scheduled for the next rescan.

See the full security report

Quality evaluation

Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

SKILL.md in the repositoryCheckingdocs/skills/sam3d-gso-manual-registration/SKILL.md

Popularity

Repository stars: 228
Repository forks: 22
7-day trend score: 0

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

Claude Code
Cursor
GitHub Copilot
Windsurf
Codex
Gemini CLI
Cline
Amp
Roo Code
Goose
Continue
Kilo Code
Junie
Trae
OpenHands
Qwen Code
AI Agent
Tabnine Chat
Sourcegraph Cody
Amazon Q
Firebase Genkit
Sweep
Devin
Phind
Mentat
Claude Desktop
Aider
GPT Pilot
Plandex
bolt.new
v0
Replit Agent
Anse
TypingMind
BetterChatGPT
LibreChat
Warp AI
Pieces
Fig AI

Embed badge

[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/kaichen-z/stream3d/sam3d-gso-manual-registration/badge)](https://verified-skill.com/skills/kaichen-z/stream3d/sam3d-gso-manual-registration)

Something wrong with this skill or its page? Tell us.