openbmb/minicpm/minicpm5-finetune-ms-swift

Claude Code and agent skill by openbmb/minicpm

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Fine-tune MiniCPM5-1B or MiniCPM5-2B with ms-swift or Megatron-SWIFT. Use when the user mentions "ms-swift", "swift sft", "swift rlhf", or "megatron sft". MiniCPM5-1B uses the PyPI release with `--template minicpm5`; MiniCPM5-2B requires `ms-swift>=4.6.0.dev0` with `--template minicpm5_2b`. Both use `--model_type llama`.

Install

npx vskill@latest install openbmb/minicpm/minicpm5-finetune-ms-swift

SKILL.md

The same SKILL.md was published earlier as openbmb/minicpm/minicpm5-finetune-ms-swift.

Fine-tune MiniCPM5-1B and MiniCPM5-2B with ms-swift

Select the version and template that match the model:

Modelms-swift versionArguments
MiniCPM5-1BPyPI 4.5.3--model_type llama --template minicpm5
MiniCPM5-2B>=4.6.0.dev0--model_type llama --template minicpm5_2b

Both models require transformers>=5.6.

ms-swift 4.x renamed `--train_type` to `--tuner_type`. Use --tuner_type lora for LoRA training.

Required input

VarExampleDefault
BASE_MODELopenbmb/MiniCPM5-2Brequired; openbmb/MiniCPM5-1B is also supported
TEMPLATEminicpm5_2buse minicpm5 for MiniCPM5-1B
DATApath to messages-format jsonlrequired
OUTPUT_DIR./runs/minicpm5_swiftrequired
GPU_ID00

Each line of DATA: {"messages": [{"role":"...","content":"..."}, ...]}.

Steps

1. Install (once)

For MiniCPM5-1B:

bash
pip install "ms-swift==4.5.3" "transformers>=5.6"

For MiniCPM5-2B:

bash
git clone https://github.com/modelscope/ms-swift.git
cd ms-swift
git checkout 654e24f17b5d9f40ed4b9ee4c56a723320244db5
pip install -e .
pip install "transformers>=5.6,<5.17"

For Megatron-SWIFT:

bash
pip install mcore-bridge==1.6.4
2. Train (LoRA SFT)
bash
CUDA_VISIBLE_DEVICES=${GPU_ID} swift sft \
--model "${BASE_MODEL}" \
--model_type llama \
--template "${TEMPLATE}" \
--tuner_type lora \
--dataset "${DATA}" \
--output_dir "${OUTPUT_DIR}" \
--num_train_epochs 2 \
--per_device_train_batch_size 4 \
--gradient_accumulation_steps 4 \
--learning_rate 2e-4 \
--lora_rank 16 --lora_alpha 32 --lora_dropout 0.05 \
--target_modules q_proj k_proj v_proj o_proj gate_proj up_proj down_proj \
--max_length 4096 \
--warmup_ratio 0.03 \
--bf16 true \
--logging_steps 10 \
--save_steps 200

Set TEMPLATE=minicpm5 for MiniCPM5-1B or TEMPLATE=minicpm5_2b for MiniCPM5-2B.

3. Validate

Loss should decrease over the first few hundred steps:

{'loss': 4.52, 'token_acc': 0.26, 'epoch': 0.04}
{'loss': 3.57, 'token_acc': 0.35, 'epoch': 1.00}

Adapter is at ${OUTPUT_DIR}/v0-${TIMESTAMP}/checkpoint-${STEP}/.

Merge for serving

bash
swift export \
--model "${BASE_MODEL}" \
--adapters "${OUTPUT_DIR}/v0-${TIMESTAMP}/checkpoint-${STEP}" \
--merge_lora true \
--output_dir ./minicpm5-swift-merged

The merged model is a regular LlamaForCausalLM and serves with any minicpm5-deploy-* skill.

Full SFT / DPO / RLHF

Same flag surface, just swap the trainer:

bash
# Full SFT
swift sft --tuner_type full ...
# DPO
swift rlhf --rlhf_type dpo \
--model "${BASE_MODEL}" --model_type llama --template "${TEMPLATE}" \
--dataset preference.jsonl \
--output_dir ${OUTPUT_DIR} ...
# Megatron-SWIFT SFT
megatron sft \
--model "${BASE_MODEL}" \
--model_type llama \
--template "${TEMPLATE}" \
--dataset "${DATA}" \
--finetune true \
--output_dir "${OUTPUT_DIR}"

Set TEMPLATE=minicpm5 for MiniCPM5-1B or TEMPLATE=minicpm5_2b for MiniCPM5-2B.

For complete training options and configuration details, see the ms-swift command-line parameters and the Megatron-SWIFT quick start.

Multi-GPU

bash
NPROC_PER_NODE=8 swift sft \
--model "${BASE_MODEL}" --model_type llama --template "${TEMPLATE}" \
--tuner_type lora --deepspeed default-zero2 \
...

Common pitfalls

  • `Failed to automatically match model_type`: add --model_type llama.
  • `Failed to automatically match template_type`: use --template minicpm5 for MiniCPM5-1B or --template minicpm5_2b for MiniCPM5-2B.
  • `minicpm5_2b is not registered`: the active Python environment is using an older ms-swift release. Install ms-swift>=4.6.0.dev0; the source installation above pins a known revision for reproducibility.
  • Conflict with LLaMA-Factory in same env: LLaMA-Factory pins transformers==4.52, ms-swift wants the latest (currently transformers ≥5.6). Use separate venvs, or set PYTHONNOUSERSITE=1 to ignore user-site transformers.

Reference

`docs/finetune/ms_swift.md`

Security

  • Security scanPassed, 90 of 100Pattern checks on the skill's files.
  • Model reviewPassed

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

Repository stars
9,518
Repository forks
625
7-day trend score
46

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/openbmb/minicpm/minicpm5-finetune-ms-swift/badge)](https://verified-skill.com/skills/openbmb/minicpm/minicpm5-finetune-ms-swift)

Something wrong with this skill or its page? Tell us.