sands-lab/focus/speculative-naming

Speculative Decoding — Naming Conventions

Apply this skill when adding, renaming, or reviewing identifiers in speculative decoding code (anything under python/sglang/srt/speculative/, related attention backends, scheduler accumulators, IPC fields, observability metrics, or CLI flags).

Rule 1 — Verb form, drop -ed

Use the verb form accept everywhere. Don't use the past-participle form accepted.

Rule 2 — The extra/bonus token is bonus_token / bonus_tokens

The "+1" token that the target model always emits in addition to verifying drafts is the bonus token. Use bonus_token / bonus_tokens per Rule 7.

req.output_ids (the full output history of a request) is unrelated and stays as is.

Rule 3 — accept includes bonus; correct excludes bonus

The semantic distinction lives in the verb, not the noun. Don't enumerate noun pairs.

Pair with whatever noun fits the data (tokens, drafts, indices, …). No required pairing, but preferred default nouns: accept_tokens and correct_drafts — correct semantically describes drafts (what got verified), accept describes the resulting token sequence (incl. bonus).

Exception: accept_rate / accept_length follow paper convention

These two metric names are entrenched in the spec-decoding literature and in external-facing fields (meta_info, Prometheus). Their semantics are paper-defined, not Rule-3-defined:

Internal counters still follow Rule 3 strict semantics: num_correct_drafts (no bonus), num_accept_tokens (with bonus).

Rule 4 — num_ for counts; _ct for counters; _rate for rates; no prefix for IDs

Each form has its own marker. Never mix (no num_X_ct, no num_accept_rate).

Rule 5 — Drop redundant _token_id / _token_ids suffix in spec scope

_id / _ids and _token / _tokens are both fine. But don't combine — _token_id / _token_ids is redundant inside spec decoding, because spec code only ever deals with vocab integers.

The semantic differs by scope:

Renames

Rule 6 — Singular vs plural

Plural for any non-scalar tensor ([bs]-shaped, flat, or multi-dim); singular only for scalars (kernel tl.load results, single-int locals). Applies to all spec-decoding tensors (tokens, indices, etc.).

accept_tokens: torch.Tensor     # [total_accepted] flat - plural
accept_indices: torch.Tensor    # [bs, num_draft_tokens] - plural
draft_tokens: torch.Tensor      # [bs * num_draft_tokens] flat - plural
bonus_tokens: torch.Tensor      # [bs] - plural
accept_token = tl.load(...)     # int32 scalar in a kernel iteration - singular
bonus_token = tl.load(...)      # int32 scalar inside a kernel - singular

Out of scope (these names stay as is)

These rules apply to spec-decoding-specific identifiers. Pre-existing or framework-level names are kept.

• PyTorch / ecosystem: seq_lens, extend_seq_lens, cu_seqlens_q
• Framework / multimodal vocab: image_token_id, pad_token_id, eos_token_id, mask_token_id, hot_token_id, bos_token_id, topk_id
• Request-level state: req.input_ids, req.output_ids, req.origin_input_ids, next_token_ids (model_runner.sample output)
• Frozen C++ kwargs: accept_token_num (sgl-kernel)
• Non-token IDs: req_id, gpu_id, layer_id, program_id
• `_len` / `_lens` names: num_X is preferred for counts (Rule 4), but _len / _lens names are acceptable. Triton kernel params in particular often use _lens / _len to align with the PyTorch ecosystem (seq_lens, cu_seqlens_q). Rule 1 still requires the -ed-less form (accept_length OK, accepted_length not).