openbmb/minicpm/minicpm5-deploy-arclight

Claude Code and agent skill by openbmb/minicpm

Trust tier 2
  • community
LLM review pending
Source check pending

Checking that the SKILL.md file is still in the repository.

Install

npx vskill@latest install openbmb/minicpm/minicpm5-deploy-arclight

SKILL.md

The same SKILL.md was published earlier as openbmb/minicpm/minicpm5-deploy-arclight.

Deploy ArcLight from Source (CPU)

ArcLight is a lightweight C/C++ LLM inference framework for unified-memory systems. The recommended path is to build from source, then run a GGUF model with al-gen, al-chat, or al-ppl.

Required input

VarExampleDefault
MODEL/path/to/MiniCPM5-2B-Q4_0.ggufrequired
PROMPT"Hello!""Hello!"
THREADS4choose for the target CPU
NUMA_MODEnone or tpnone for first run
NODES1, 2, 41 with NUMA_MODE=none
MAX_GEN256256

Steps

1. Build from source
bash
git clone https://github.com/OpenBMB/ArcLight.git
cd ArcLight
cmake -B build -DARCLIGHT_BACKEND=AUTO
cmake --build build --config Release -j 32

Use ARCLIGHT_BACKEND=AUTO by default. Set it explicitly only when needed:

  • X86: force the x86 backend
  • NEON: force the ARM NEON backend
  • NONE: build without architecture-specific backend code
2. Prepare a GGUF model

ArcLight uses GGUF checkpoints from llama.cpp. The nnml backend only loads f32 / f16 / q4_0 / q8_0 / q6_K / q8_K tensor types — `Q4_K_M` is not supported.

Supported model families: MiniCPM5-2B, Qwen3, Llama2.

For first validation use the released Q8_0 (openbmb/MiniCPM5-2B-GGUF), or quantize an unreleased Q4_0 yourself from the F16:

bash
huggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-F16.gguf --local-dir .
llama-quantize ./MiniCPM5-2B-F16.gguf ./MiniCPM5-2B-Q4_0.gguf Q4_0
3A. One-shot generation
bash
./build/al-gen \
--model "${MODEL}" \
--prompt "${PROMPT}" \
--numa none --nodes 1 \
--threads ${THREADS} \
--max_length 4096 \
--max_gen ${MAX_GEN}
3B. Interactive chat
bash
./build/al-chat \
--model "${MODEL}" \
--numa none --nodes 1 \
--threads ${THREADS} \
--max_length 4096 \
--max_gen ${MAX_GEN}

To seed the first turn:

bash
./build/al-chat \
--model "${MODEL}" \
--prompt "${PROMPT}" \
--numa none --nodes 1 \
--threads ${THREADS}
3C. Perplexity
bash
./build/al-ppl \
--model "${MODEL}" \
--prompt "Good morning, Miss Lee!" \
--numa none --nodes 1 \
--threads ${THREADS}

Many-core CPU / NUMA

Start with single-node mode for correctness:

bash
./build/al-gen \
--model "${MODEL}" \
--prompt "${PROMPT}" \
--numa none --nodes 1 \
--threads ${THREADS}

Use cross-NUMA tensor parallelism on many-core machines:

bash
./build/al-gen \
--model "${MODEL}" \
--prompt "${PROMPT}" \
--numa tp --nodes ${NODES} \
--threads ${THREADS}

Rules:

  • --numa none requires --nodes 1.
  • --numa tp requires --nodes N where N > 1.
  • In the current version, NODES should be a power of 2.
  • Choose THREADS so it can be evenly divided across NODES.
  • --numa pp is reserved for future pipeline parallelism and is not implemented.

Optional memory buffers

If allocation fails or the model is larger, pass manual buffer sizes:

bash
./build/al-gen \
--model "${MODEL}" \
--prompt "${PROMPT}" \
--numa none --nodes 1 \
--threads ${THREADS} \
--w_gb 4 --a_gb 8 --kv_gb 2 --work_gb 2

Meaning:

  • --w_gb: weight buffer size
  • --a_gb: activation buffer size
  • --kv_gb: KV cache buffer size
  • --work_gb: temporary workspace size

Increase --kv_gb for longer --max_length. Increase --w_gb for larger models.

Validate

Run:

bash
MODEL=/path/to/MiniCPM5-2B-Q4_0.gguf
THREADS=4
./build/al-gen \
--model "${MODEL}" \
--prompt "1+1=?" \
--numa none --nodes 1 \
--threads ${THREADS} \
--max_gen 64

The reply should contain 2 or a short explanation that evaluates to 2.

Common pitfalls

  • Program aborts in single-node mode: use --numa none --nodes 1.
  • Tensor parallel mode fails: use --numa tp --nodes N with N > 1; in this version, N should be a power of 2.
  • Pipeline parallelism fails: --numa pp is not implemented yet.
  • Model loading fails: verify the checkpoint is GGUF and from a supported model family.
  • Out of memory: increase --w_gb, --a_gb, --kv_gb, or --work_gb; longer contexts usually need a larger KV cache.
  • Poor CPU throughput: check --threads, NUMA layout, and thread-core bindings; use --print_binding 1 --print_perf 1 for diagnostics.

When NOT to use

  • Need CUDA server inference -> use a GPU-oriented runtime instead.
  • Need Apple Silicon MLX inference -> use an MLX deployment path instead.
  • Need a desktop GUI -> use a GUI runtime that supports GGUF models.
  • Need pipeline parallelism -> wait for ArcLight --numa pp support.

Reference

Security

  • Security scanPassed, 100 of 100Pattern checks on the skill's files.
  • Model reviewNot runA model review is scheduled for the next rescan.

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

Repository stars
9,518
Repository forks
625
7-day trend score
46

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/openbmb/minicpm/minicpm5-deploy-arclight/badge)](https://verified-skill.com/skills/openbmb/minicpm/minicpm5-deploy-arclight)

Something wrong with this skill or its page? Tell us.