infinigence/apxinf/model-port-workflow

Model Port Workflow

Required reading

Resolve the ApxInf checkout/worktree being modified first. All repository paths below refer to that checkout, even when this skill is loaded through a global symlink pointing at another checkout. Read its AGENTS.md for artifact placement, then read the relevant documents from that same source revision:

1. `doc/porting-workflow.md` for the complete

evidence and acceptance sequence.

1. `doc/adding-a-new-model.md` before creating

or changing model-layer code. Follow its separate-directory and YAGNI rules.

1. `doc/model-layer-architecture.md`

for the canonical module names, responsibility table, dependency direction and current family/option limits before deciding code placement.

1. `doc/model-execution-wiring.md` before

composing model/Blocks calls, preparing a runner, or deciding that optimized coverage is missing. Its accelerator acceptance section defines the native GPU and required capture gates. For Apple Silicon MLX, also read `doc/mlx-backend.md` for the separate compiled execution/storage contract; never report MLX compile as CUDA Graph replay.

1. `doc/adding-new-kernels.md` whenever

operator, dtype, layout, shape, or hardware coverage is missing.

Read each selected document completely. Treat them as instructions, not background material.

Assign module ownership before implementation

Use doc/model-layer-architecture.md#current-module-names-and-responsibilities in the selected checkout as the source of truth. The port plan must name the actual family-local owner for each responsibility:

For PI0.5 these are model/, model_runner/, weights/, load.rs and the existing policy. ModelVariant is private precision dispatch; LoadedModel remains the common loader result. StepModulation names computed per-step scale/shift/gate tensors. Use current source symbols for unmigrated families. A single-implementation model does not need a variant enum, generic Blocks or additional wrapper just to match PI0.5. Keep model and weights independent of runner/capture types; keep concrete precision dispatch out of the runner.

New accelerator ports must satisfy doc/model-execution-wiring.md#accelerator-port-acceptance: ApxInf-native GPU computation, explicit public transfers, and required fixed-profile capture. For an MLX target, the backend-specific compiled execution contract applies; its explicit IO, scope, warm-resource and eager/compiled checks replace neither numerical validation nor device residency. CUDA capture requirements remain unchanged. External engine and CPU paths are private references only. A required missing device/capture path is a blocker, not performance debt. Existing-family status and historical GPU evidence do not prove a new family's acceptance.

For the implemented native MLX selections, Qwen3 bf16-compiled prepares local norm/RoPE and decoder blocks, while mixed-w8 uses local functions, scoped W8 weights and the Q/K Metal fusion. MiniCPM5 bf16-compiled prepares the complete decode step with packed residual/norm; its explicit dspark variant adds the compiled draft chain and target verification schedule. Use the exact scope/profile table in doc/mlx-backend.md. None of these is CUDA Graph replay or evidence for an unrelated model family.

Workflow

1. During preflight, reuse the task's confirmed inputs and execution mode; use

hands-off for an implementation request unless the user chooses hands-on. Ask only for missing choices that affect the port. Offer discovered official GitHub source and Hugging Face checkpoint candidates; offer detected hardware defaults only for local Thor or Orin. Then fix the reference revision, checkpoint identity, target, precision, representative inputs, tolerances, performance goals, and public API. Do not silently pick user choices: record the confirmed or explicitly authorized tuple. Completion: every requested tuple and acceptance threshold is explicit.

1. Run the reference privately and capture deterministic inputs, outputs, and

diagnostic tensors. Completion: the reference loads and repeated captures are attributable to the same source, environment, weights, and stochastic inputs.

1. Inventory semantics and weight transformations. Completion: every required

computation is understood independently of its framework operator name.

1. Create an execution ledger from reference semantics through safe device calls.

Inspect maintained model/Blocks implementations and fused interfaces before the portable backend trait. Account for tensor lifetime, reusable KV/state, workspace, host traffic, and graph eligibility. Define the intended whole graph or Vision/Language/Action graph boundaries, stable buffers, input-update mechanism, and capture blockers. Completion: every graph row resolves to an ApxInf-native implementation, a private scaffold with a named device exit criterion, or a concrete blocker. For MLX, record local, decoder-block and whole-step compile scopes separately; identify explicit dynamic state, prepared shapes and bounded lazy outputs. Inspect the safe array/fusion seam in apxinf-mlx; keep raw FFI and unsafe custom kernels there, without a universal model/runtime wrapper. Use metal_<family> for native Metal composition and registration. The evolving cross-backend contract is not a prerequisite: preserve the model's numerical recipe and measured performance without introducing per-op trait wrappers.

1. Classify fused and primitive coverage. If a real gap exists, follow

adding-new-kernels.md, then replay the returned implementation against the original references. A CPU layer implementation is a named correctness scaffold with an exit criterion. On an accelerator target, replace every repeated hot-path scaffold with a safe device path; build cost and passing end-to-end values do not turn host round trips into deliverable performance debt.

1. Create crates/apxinf-model/src/<model>/. Start with a self-contained

implementation. Inspect and copy a close model when useful, but do not import or modify another model-family directory for the new architecture. Defer shared extraction to a separate review after repeated maintained implementations prove the seam. Run scripts/check_model_family_boundaries.sh before review.

1. Follow doc/adding-a-new-model.md#registration-and-public-integration.

Use LlmTrait and LoadedModel::text for LLM/VLM, or VlaRuntime on the family runner and LoadedModel::Vla for VLA. Wire Rust registration; for VLA, also wire Python policy discovery and reuse apxinf.ModelRunner / apxinf_py.ModelRunner and model_runner injection. Check the common loader's current model_variant admission and PI0.5-only config overrides before extending options. When adopting explicit preparation, read the implemented PI0.5 section of doc/model-lifecycle/lifecycle.md. Test mode/fallback, stale-plan rejection, tuning-store replacement/generation, compatible run without capture/autotune, input/RNG rebinding and output/resource lifetime. Default RuntimeManaged status and unsupported policy methods are not readiness. For cuda-new, verify GEMM and Attention cold-cache tuning, fresh-process recipe reuse with autotune disabled, and the public --autotune/load-option wiring. Run default-path tests from a task artifact directory to isolate generated configs/tuning/.../*.recipe files; never depend on legacy JSON presence. Completion: registry loading, native contract, Python policy/public inference and family-specific GPU evidence agree in the same change. Keep LlmTrait::forward full [seq_len,vocab_size]. Use prefill for a last-row generation optimization and the request hook for speculative schedules. Test the checkpoint chat template and decoding semantics through the public CLI: --chat-options '{"enable_thinking":false}' controls the template; --keep-special-tokens preserves semantic special tags (also automatic when chat options contain tools). Do not strip MiniCPM's <function>/<param> tokens from tool XML. An experiment's /no_think suffix is frozen replay data, not a universal user-prompt requirement.

1. Verify operators, transformations, intermediate checkpoints, eager and

captured inference, host-transfer audit, public serving/policy integration, and requested performance. Prove that tensor computation between public input upload and public output transfer stays on GPU, and verify eager versus captured parity, repeated replay, changed-input propagation, stable addresses, and capture-safe workspace use. For a VLA, verify the whole-model graph or all three minimum Vision/Language/Action graphs. Report functional acceptance separately from optimization status (target met, best effort with performance debt, or blocked). Performance is best effort unless explicitly declared a release gate, but applicable existing optimized paths must be investigated. For MLX, check compiled parity and explicit-state publication, rather than treating fixed-address capture as its mechanism. Keep changed-input, retained-output and load/reset/drop tests. Use Compiled::prepare for admitted shapes and reject MLX_DISABLE_COMPILE even when set to 0; do not double-execute the runtime path. The second prepared invocation must reuse its trace. Bridge upload/download counters describe explicit host IO, trace counters describe Rust tracing, and MLX allocator statistics are process-wide; none alone proves absence of all native JIT, physical transfers or memory growth. Bind source, binary, weights, original complete answers and every measured report; quantized variants need both their precision-matched source comparison and the model-quality floor. Check text preparation_status() state and its actual prepared lengths/scopes: RuntimeManaged is not proof of readiness; invalidated/unprepared states must not report successful compilation. Keep this pure-data text interface independent of VLA preparation and the family-owned executable resources.

1. Prepare a product-only diff. Store generated captures, reports, temporary

adapters, replay scripts, generated plans, and agent state in the ignored <project-root>/devlocal/<feat-name>/ directory according to AGENTS.md. Reuse existing external checkpoints and reference checkouts in place. Do not open a model-port review until correctness scaffolds are removed, required capture/replay passes, and the linked accelerator contract passes. Update module/capability docs, examples and relevant skills in the same PR; validate referenced source paths and keep historical results revision-labelled.

Stop conditions