Skill v2.3.6
currentAutomated scan100/100name: ml description: Applies ML/DL research engineering discipline — reproducible project layout, leakage-safe dataset construction, and a training-discipline ladder — to classical ML, deep learning, fine-tuning, and vision work. Use when scaffolding a new ML project, asked to "build a dataset" or "데이터셋 구축", running or reviewing a "train a model" experiment, or building a "vision model" pipeline (augmentation, detection, segmentation). Not for per-file Python discipline (typing, TDD loop) — use principle-programming. Agent behavior (prompts, tools, agent evals) is outside this package. GPU/CUDA environment setup or shared-host job launch belongs to gpu. metadata: version: 2.3.6
ml
Run ML/DL research and engineering work under one discipline: reproducibility first, evaluation honesty second, novelty third. Label preliminary or non-reproducible observations honestly; do not present test-tuned numbers as independent evidence or claim an improvement without a comparable baseline. This skill is an index — shared rules live here, per-topic methods live in references/; load the matching reference before touching a project layout, a dataset, or a training run.
Output contract
Produce the requested modeling artifact or review with the relevant dataset, configuration, source identity, observed outcomes, and verification limits. Preserve the incumbent environment and distinguish intended commands from actual runs. Missing launch authority, unresolved input identity, or a failed required recovery test blocks that launch, not unrelated read-only diagnosis. Report missing evidence explicitly; never invent a metric, successful run, or reproducibility receipt.
Task gate — run first, every time
Identify the task type before writing dataset, training, or vision code. Rows stack — load every reference whose row matches, not only the first one that fits:
| Task | Read | Notes | |
|---|---|---|---|
| New project or explicitly scoped package/layout migration | references/project-layout.md | Use its uv/pyproject.toml/src/ recipe as a greenfield default; preserve an established project's lockfile, environment, and layout during feature work. | |
| Dataset construction, ingestion, splitting, labeling, or versioning | references/datasets.md | Stacks with training.md and/or vision.md whenever the dataset also feeds a training run or is image/video data. | |
| A training run for any model class, including standalone LLM fine-tuning (SFT, LoRA) | references/training.md | ||
| A vision task (image/video pipeline, augmentation, detection/segmentation) | references/vision.md | Load in addition to training.md — vision rules layer on top of the general training ladder, not replace it. | |
| A run that will execute on a GPU host — CUDA/build selection, VRAM budgeting, shared-machine or HPC launch | Load the gpu skill first | gpu proves the environment and gates the launch; the references here still own the methodology once the environment is proven. | |
| Building or changing agent behavior — prompts, tools, a tool-use loop, agent evals | Stop here | Follow the target project's agent-behavior and evaluation contract; this is not model training. |
Example: labeling and splitting an image dataset that will then be trained on matches three rows at once — load datasets.md + training.md + vision.md together, not vision.md alone.
Core rules
- Reproducibility receipt. Bind a compared result to its source revision and frozen current-content identity, configuration, data manifest, actual invocation, and observed outcomes. Qualify missing evidence rather than inventing it.
references/training.mdowns the full tracking contract, including reported generative evaluations. - Split before you fit anything. Train/validation/test separate before any statistic is computed from the data — scaling, imputation, vocabulary, augmentation parameters.
references/datasets.mdgives the three leakage classes and their detection commands. - A baseline exists before a novel approach is judged. "Better than the majority-class/linear/frozen-pretrained baseline," not "better than nothing."
references/training.mdcovers the discipline ladder in full. - Reserve test data for frozen final evaluation, never model selection or tuning; disclose repeated evaluation rather than calling a reused test set fresh evidence.
- Mutable library/runtime facts. For framework APIs, installation, compatibility, or runtime behavior, consult the library's official documentation first. Disclose conflicts; a more-specific local contract or reproducible evidence for the matching library version and platform may override general or stale documentation. If unresolved, leave it unknown and stop or use the applicable safe fallback — never invent a capability or command.
Requirements
- Python: for a greenfield project,
uvis a default for dependency locking and running;pandas/polars,pandera(or an equivalent schema checker), andscikit-learn/torch/jaxare examples chosen to fit the task. In an established project, use its existing locked environment and installed tooling. - Runtime and framework maintenance: record the applicable official Python documentation,
python3 --version, the incumbent lockfile, the installed framework version from its documented in-environment probe, and the selected framework's official release source. When probe or release evidence shows a selected runtime or framework's runtime form changed, recheck the official documentation, rerun affected package evaluations for datasets, training, and GPU work, update the recipe if needed, then bump this package's version and record the reason in the commit message before reporting results. git,grep,find,awk,sha256sum(shasum -a 256on macOS) for the detection commands in each reference.- A config format the team already uses for one-run-one-config (examples assume YAML).
Common rationalizations
| Rationalization | Reality | |
|---|---|---|
| "It's just a quick experiment script, skip the project layout." | A throwaway exploration still needs a smoke test and must respect the incumbent environment and layout; the full baseline, comparison, and variance ladder applies when its result will be reported or used for a decision. | |
| "I already know the data is clean, skip split-before-fit." | Leakage is invisible in code, visible only in an eval number that quietly stops meaning anything. | |
| "The new architecture is obviously better, no baseline needed." | "Obviously better" without a baseline number is an opinion, not a result. | |
| "I used the test metric to choose the next candidate, but it is still held out." | Use validation data for selection. A test-informed candidate no longer has independent evidence from that test set; disclose the reuse and evaluate accordingly. | |
"This is mostly a fine-tuning job, so ml covers it" (even though it calls tools). | If the feature calls tools, reasons over retrieved context, or drives multi-step LLM behavior, follow the target project's agent-behavior contract regardless of what else it touches. |
Red flags
- A greenfield project with no locked environment or importable training code, or an unrelated feature that rewrites an established project's packaging.
- A preprocessing/scaling/vocabulary step that runs before the split, or on the concatenation of all three.
- A "novel" result reported with no baseline number in the same table.
- A compared run whose actual code/configuration inputs cannot be recovered from its recorded revision and frozen snapshot.
- A robust-improvement claim without uncertainty evidence appropriate to the task, or invented variance for a single run.
- An augmentation transform present in the validation or eval data loader.
Boundaries
Not for wrapping a trained model behind a serving API — load principle-backend — or for suite-level test-architecture decisions — load principle-testing. GPU/CUDA environment, compatibility, and shared-host launch preflight are gpu's domain — a GPU training run loads gpu first, then this skill. Double-check the agent-behavior boundary from the task gate: "the model calls a tool" or "the pipeline reasons over retrieved text" is agent work even when it also touches a model file.
Verification
- [ ] The task gate identified the task type and the matching reference was read before writing code — or the task was recognized as agent work outside this package.
- [ ] The established project's locked environment and layout were preserved, or greenfield code has a locked environment and is importable.
- [ ] Every fitted statistic (scaler, vocabulary, augmentation parameter) is fit on the train split only.
- [ ] A baseline number exists in the same report as any novel-approach number.
- [ ] Each reported result has a reproducibility receipt: the code revision, configuration, data manifest, and recorded outcome identify what ran.
- [ ] Test data did not drive tuning or model selection, and any frozen evaluation replay is disclosed.
- [ ] Claims report the actual sampling/repeat count and appropriate uncertainty; limited evidence is labeled without a universal seed quota.