openbmb/minicpm/minicpm5-deploy-sglang

Claude Code and agent skill by openbmb/minicpm

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for "SGLang", "RadixAttention", "prefix cache", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.

Install

npx vskill@latest install openbmb/minicpm/minicpm5-deploy-sglang

SKILL.md

The same SKILL.md was published earlier as openbmb/minicpm/minicpm5-deploy-sglang.

Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang

OpenAI-compatible server with RadixAttention prefix cache. Best fit for tool calling, batched eval pipelines, and high-concurrency serving.

Required input

VarExampleDefault
MODEL_PATHopenbmb/MiniCPM5-2B or openbmb/MiniCPM5-1Brequired;
PORT3000030000
GPU_ID00
CTX_LEN131072 (128 K)131072
MEM_FRAC0.850.85
TOOL_PARSERminicpm5minicpm5; use auto if you want template detection

Steps

1. Install (once)
bash
pip install "sglang[srt]>=0.5.16" # latest, requires CUDA 13.x driver
# pip install "sglang==0.5.6.post3" # fallback for CUDA 12.x driver hosts

For tool calling, install from `main` — the minicpm5 parser (PR #25600, merged 2026-05-22) is not in any pip release yet (v0.5.12.post1 was branched earlier). Plain chat works on the pip release; only --tool-call-parser minicpm5 needs main:

bash
pip install "git+https://github.com/sgl-project/sglang.git@main#subdirectory=python"
2. Recommended runtime env vars
bash
export VLLM_WORKER_MULTIPROC_METHOD=spawn
export SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1
export SGLANG_DISABLE_CUDNN_CHECK=1
3. Launch
bash
CUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \
--model-path "${MODEL_PATH}" \
--served-model-name MiniCPM5-2B \
--dtype bfloat16 \
--context-length ${CTX_LEN} \
--mem-fraction-static ${MEM_FRAC} \
--tool-call-parser ${TOOL_PARSER} \
--host 0.0.0.0 \
--port ${PORT}

Wait for The server is fired up and ready to roll! .

Speculative decoding (DSpark): we also release MiniCPM5-2B-DSpark, a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:

bash
CUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \
--model-path "${MODEL_PATH}" \
--served-model-name MiniCPM5-2B \
--dtype bfloat16 \
--context-length ${CTX_LEN} \
--mem-fraction-static ${MEM_FRAC} \
--tool-call-parser ${TOOL_PARSER} \
--host 0.0.0.0 \
--port ${PORT} \
--speculative-algorithm DSPARK \
--speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \
--speculative-dspark-block-size 7
4. Validate
bash
curl http://localhost:${PORT}/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "MiniCPM5-2B",
"messages": [{"role": "user", "content": "1+1=?"}],
"temperature": 1.0, "top_p": 0.95, "max_tokens": 64,
"chat_template_kwargs": {"enable_thinking": true}
}'

Expected: choices[0].message.content contains "2".

Tool calling

SGLang includes a native MiniCPM5 XML tool-call parser. Keep TOOL_PARSER=minicpm5 (or auto) when launching, then send an OpenAI-style tools request:

bash
curl http://localhost:${PORT}/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "MiniCPM5-2B",
"messages": [{"role": "user", "content": "What is the weather in Beijing?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"tool_choice": "auto",
"temperature": 1.0, "top_p": 0.95,
"max_tokens": 256
}'

Offline / batched (Engine API)

python
import sglang as sgl
llm = sgl.Engine(model_path="${MODEL_PATH}", tp_size=1, mem_fraction_static=0.8, context_length=131072)
outputs = llm.generate(
["用一句话解释什么是 GQA。"],
sampling_params={"temperature": 1.0, "top_p": 0.95, "max_new_tokens": 1024,
"skip_special_tokens": False},
)
print(outputs)

Common pitfalls

  • `GLIBCXX_3.4.31 not found` (only if you hit it — not universal): conda Python ships an older libstdc++ than the SGLang wheel was built against. Force-load the system one: LD_PRELOAD=/lib/x86_64-linux-gnu/libstdc++.so.6 python -m sglang.launch_server ...

When NOT to use

  • Production serving without batch-eval needs → minicpm5-deploy-vllm is simpler
  • One-shot Python → minicpm5-deploy-transformers
  • No NVIDIA GPU → minicpm5-deploy-llama-cpp / minicpm5-deploy-mlx

Reference

`docs/deployment/sglang.md`

Security

  • Security scanPassed, 95 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

Repository stars
9,518
Repository forks
625
7-day trend score
46

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/openbmb/minicpm/minicpm5-deploy-sglang/badge)](https://verified-skill.com/skills/openbmb/minicpm/minicpm5-deploy-sglang)

Something wrong with this skill or its page? Tell us.