castform-ai/benchmax/train-sft

Train with SFT

Use this when the user already has the completions they want the model to imitate — support transcripts, labeled chat data, tool-call traces, input→output pairs. There is no environment, tool loop, reward, or validate stage: the RL loop in this project's other skills does not apply. If the task needs the model to discover good behavior against a scorer, use the RL skills instead.

The whole flow

from benchmax.sft import SftDataset, SftDatasetError
from castform.platform import SftTrainingConfig, TrainerClient, upload_sft_assets
train = SftDataset.from_jsonl("train.jsonl")      # or SftDataset.from_rows(rows)
uploaded = upload_sft_assets(dataset=train, run_name="support-sft")
run_id = TrainerClient().launch_sft_run(
    assets=uploaded,
    name="support-sft",
    config=SftTrainingConfig(num_epochs=1, learning_rate=1e-5, seed=42),
)

Construction is all-or-nothing: SftDataset either satisfies the whole benchmax-sft-v1 contract or raises SftDatasetError with every issue, ordered and line-aware. Fix the data the diagnostics point at — never pre-filter rows silently or patch around individual issues without telling the user.

Row contract (one JSON object per line)

{
  "messages": [
    {"role": "user", "content": "What is 2 + 2?"},
    {"role": "assistant", "content": "4", "weight": 1}
  ],
  "tools": [],
  "metadata": {"id": "optional producer identity"}
}

• system/user: exactly role + non-empty string content.
• assistant: optional string-or-null content, optional non-empty

tool_calls, optional integer weight 0 | 1 (omitted means 1). Each assistant turn needs content or a tool call; each row needs at least one assistant turn with effective weight 1.

• tool results: exactly role + string content + non-empty tool_call_id;

every tool call gets exactly one result, in declaration order, before the next non-tool message. Tool definitions are OpenAI function shapes; function.arguments must decode as a JSON object.

• Everything else is rejected: images/audio/multimodal parts, fractional

weights, legacy prompt/completion keys, unknown fields, duplicate JSON keys, rows over 1 MiB, more than 1024 messages.

Masking: set weight: 0 on assistant turns that are context, not target (earlier drafts, retrieved answers, another model's output). Only weight-1 turns contribute to the loss.

What the user actually chooses

Every arg whose default reads unset is optional: leave it out and the platform's own value applies, so an untouched config behaves exactly as before these knobs existed. Do not set one just to restate a default.

lora_rank picks the adapter's rank; alpha is always derived as 2x and is not a knob. Rank 128 is not accepted — it trains, but serving cannot load it.

Model (Qwen/Qwen3.5-4B) and GPU topology are platform-owned — do not invent knobs for them. A row that renders past max_context_tokens tokens fails the run's preflight; trim long rows up front.

Held-out eval (optional)

Pass a second dataset to score during training:

assets = upload_sft_assets(dataset=train, eval_dataset=held_out, run_name="...")

The eval set is scored on the live weights between training steps and plotted as eval/loss beside train/loss. Rules worth knowing before you offer it:

• At most 2048 rows, under the same per-row limits as training rows. The

bound is a compute bound, not a taste one — eval runs on the training GPUs and pauses training while it does.

• It is never billed. Eval forward passes cost the user nothing, which is

also why the size and cadence are capped rather than left open.

• Cadence defaults to `save_interval`, so by default every eval lands on a

checkpoint. Set eval_interval only to make it sparser; a much denser cadence is rejected at launch.

• An eval set is part of the data identity: changing it produces a different

upload prefix, and a resumed run must keep the one it started with.

Skip it for small or exploratory runs — a held-out split costs training rows, and train/loss alone answers "is this learning at all".

Cost and consent

launch_sft_run spends GPU credits. Ask the human before calling it, every time — preparing and uploading the dataset first is free and fine. Steps per epoch ≈ rows / 4 (tiny datasets pad by repeating their first rows, so 1–3-row datasets train on repeats; prefer at least a few dozen rows). Stopping a run keeps only checkpoints already uploaded; work since the last one is lost.

If launch fails with "SFT launch is not enabled", the platform gate is off for this account — surface that to the user rather than retrying.

Monitor

Same as any run (view-progress skill): castform runs status <id>, and castform runs scalars <id> — watch train/loss fall, and eval/loss too when the run has an eval set. There are no rollouts or reward curves for SFT runs. The run page shows loss, dataset prefix, and config; runs with an eval set also get an eval tab, chart-only.

Reference

The canonical worked example (streaming a pinned public corpus, bounded mapping, offline tests, explicit paid --launch gate) ships in the benchmax repo under examples/sft/pii_masking/.