openbmb/minicpm/minicpm5-deploy-llama-cpp

Claude Code and agent skill by openbmb/minicpm

Trust tier 2
  • community
LLM review pending
Source check pending

Checking that the SKILL.md file is still in the repository.

Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for "llama.cpp", "llama-cli", "llama-server", "GGUF", or has no Python available.

Install

npx vskill@latest install openbmb/minicpm/minicpm5-deploy-llama-cpp

SKILL.md

The same SKILL.md was published earlier as openbmb/minicpm/minicpm5-deploy-llama-cpp.

Deploy MiniCPM5-1B and MiniCPM5-2B with llama.cpp

CPU / edge / consumer-GPU deployment via the released GGUF artifacts. The artifacts work directly with vanilla llama.cpp and every downstream runtime (Ollama / LM Studio / llama-cpp-python).

Required input

VarExampleDefault
GGUF_REPOopenbmb/MiniCPM5-2B-GGUFrequired; openbmb/MiniCPM5-1B-GGUF also works
QUANTQ4_K_M (1.56 GB, recommended) / Q8_0 (2.68 GB) / F16 (5.04 GB)Q4_K_M
NGL99 (all layers on GPU) / 0 (CPU only)99 if NVIDIA GPU, else 0
CTX8192 (default) up to 131072 (128 K)8192

Steps

1. Install llama.cpp
bash
# macOS
brew install llama.cpp
# Linux / cross-platform: pre-built binary
curl -fsSL https://github.com/ggerganov/llama.cpp/releases/latest/download/llama-cli-linux.tar.gz | tar -xz
# OR build from source:
git clone --depth=1 https://github.com/ggerganov/llama.cpp.git && cd llama.cpp
mkdir build && cd build
cmake .. -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release # CPU-only: omit GGML_CUDA=ON
cmake --build . --config Release -j $(nproc) --target llama-cli llama-server
2. Download the GGUF
bash
mkdir -p ~/minicpm5 && cd ~/minicpm5
huggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .
3a. Interactive chat (CLI)
bash
llama-cli -m MiniCPM5-2B-${QUANT}.gguf \
-n 2048 --temp 1.0 --top-p 0.95 --min-p 0.0 -ngl ${NGL} -c ${CTX}
3b. OpenAI-compatible HTTP server
bash
llama-server -m MiniCPM5-2B-${QUANT}.gguf \
--port 8080 -ngl ${NGL} -c ${CTX} --jinja
4. Validate
bash
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "MiniCPM5-2B",
"messages": [{"role":"user","content":"1+1=?"}],
"temperature": 1.0, "top_p": 0.95, "min_p": 0.0, "max_tokens": 64
}'

Expected: "2" in the reply.

In llama.cpp, the default min_p=0.05 can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set min_p=0.0.

Sampling defaults

Mode--temp--top-p--min-p
MiniCPM5-2B Think1.00.950.0
MiniCPM5-1B Think0.90.950.0
MiniCPM5-1B No-think0.70.950.0

Choosing a quant

QuantDiskQuality
F165.04 GBreference
Q8_02.68 GB~indistinguishable from F16
Q4_K_M1.56 GBsmall drop, ideal for laptops

Common pitfalls

  • Slow on CPU + large context: drop -c 131072 to -c 8192 if you don't need 128 K.

Building your own GGUF (advanced)

If you've trained your own MiniCPM5-1B or MiniCPM5-2B variant, build a GGUF with:

bash
python convert_hf_to_gguf.py /path/to/your-fp16-hf --outfile out/F16.gguf --outtype f16
llama-quantize out/F16.gguf out/Q4_K_M.gguf Q4_K_M

Trained a LoRA adapter (not a full model) and want to apply it at runtime with --lora instead of baking it in? Convert it to a GGUF adapter — see `minicpm5-finetune-gguf-lora`.

When NOT to use

  • NVIDIA GPU + want OpenAI-compatible serving → minicpm5-deploy-vllm
  • Apple Silicon native → minicpm5-deploy-mlx is faster
  • Just want one-line desktop run → minicpm5-deploy-ollama
  • Want a desktop GUI → minicpm5-deploy-lmstudio

Reference

`docs/deployment/llama_cpp.md`

Security

  • Security scanPassed, 100 of 100Pattern checks on the skill's files.
  • Model reviewNot runA model review is scheduled for the next rescan.

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

Repository stars
9,518
Repository forks
625
7-day trend score
46

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/openbmb/minicpm/minicpm5-deploy-llama-cpp/badge)](https://verified-skill.com/skills/openbmb/minicpm/minicpm5-deploy-llama-cpp)

Something wrong with this skill or its page? Tell us.