overmind-core/overmind/overmind-datasets

Overmind Datasets

For an end-to-end data-to-training request, start with model-training-engineering for the cross-stage evidence and launch checks, then use this skill for Datasets.

The calling coding agent owns interpretation, transformation authoring and semantic work. Overmind stores sources, executes retained native-agent-authored scripts, measures technical compatibility and retains versions. There is no platform Workshop agent.

Resolve the project with list_projects; account connections pass project_id on every operation. Read overmind://interface/current for the connected contract. Names are not unique: use returned dataset and cell UUIDs.

For a collection of files, use overmind dataset upload FILE1 FILE2 ... with the returned project/dataset arguments. One ordered batch (up to 100 files / 2 GiB, with per-file type limits) creates one source version and retains every original file. Resume the identical command after interruption; do not upload each file as a separate attachment unless separate versions are intended. A changed order or recipe requires a new request key. --json-rows-field applies to a single file.

Preparation lifecycle

Before dataset preparation work, inspect_dataset_workbench and resume its preparation_workflow or call start_dataset_preparation with a stable request_key. Keep it preparing across local authoring, previews and publications. Use update_dataset_preparation with state=completed and verified published output cell UUIDs only when all requested agent work is finished; use paused when stopping temporarily or cancelled when abandoning. Individual job completion does not finish preparation.

Intent and contracts

A dataset is a landed source and a linear chain of cells; every cell that ran is a version (1.0 is the source, then 1.1, 1.2, …). The dataset has an intent chosen by the user and a capability, and every version carries measured contracts (list_datasets shows the active version's):

• `train`: a messages column whose every row is a chat transcript with an

assistant turn (tools optional), or a native decision with preserved probability/ordinal targets.

• `eval`: an input on every row plus an expected_output column with at least

one reference.

• `explore`: data exploration, without automatic train/eval preparation.
• `pending`: no user purpose yet. Obtain their choice of Training, Eval or Data

exploration and set it with update_dataset; never infer intent from rows. A leftover stored ft is train; there is no ft or surface intent.

Set intent at upload (overmind dataset upload FILE --json --intent train|eval), at create (create_dataset_from_traces / _failures), or later with update_dataset(intent="train") if no version has been used. split on create_dataset_from_traces, or --split PERCENT on upload, lands one selection as a train dataset and an eval dataset with disjoint rows. There is no reingest tool and no dual-intent dataset.

Eval runs (run_evaluation) and optimizer experiments (start_optimizer) use an eval version; train and pending are refused. Fine-tuning (start_finetune) needs a train version and, for a chat model, a user-confirmed eval set; a separate eval dataset is needed only when chat judge evaluations are requested. Native decision comparisons use their frozen calibration/final plan in place of the eval set.

A use freezes the version and everything before it, and starts a new major (2.0). A used cell freezes intent: derive a separate dataset or create role-specific partitions instead of retagging a used chain. A contract is measured, never declared. Capability prompt/schema mismatches, incomplete quality reviews and train/eval overlap are advisory warnings, not technical-format errors; inspect cell warnings and readiness.quality_reason. A passing format contract is not a claim that the answers are supported by the inputs.

Sources and inspection

For a training task, retain source-bound input availability, target meaning, prediction unit, connected grouping, slices and success metric with save_training_plan. After Workshop publication and partitioning, save a superseding snapshot with exact versions. Keep evidence and uncertainty attributable; Training consumes the identities through plan-bound full runs or optional pilot experiments. See Develop models from data.

Before uploading, inspect the local file's record boundaries, fields, value types and expected record count. A JSON wrapper is not itself a dataset row: identify the intended top-level record array from the source and user request, and pass --json-rows-field FIELD. If multiple arrays have unresolved meanings, clarify which records are intended instead of guessing. Preserve the original file. After landing, compare the source cell's count and representative nested values with that inspection. A successful transfer is not proof of correct ingestion; resolve mismatches before transformation or consumer handoff, and state the coverage of verification rather than calling a sample a full-file audit.

Use start_dataset to save a brief, or create_dataset_from_traces / create_dataset_from_llm_calls for captured evidence. Local files use overmind dataset upload FILE --project-id PROJECT --json, optionally --dataset DATASET to append sources. See overmind://dataset-upload for the current transfer contract. PDF and image OCR, original bytes and extraction evidence remain platform services. OCR uses English by default and records actual languages and model identities. For another source language, call update_dataset with ocr_languages (up to four supported Tesseract codes, such as ["eng", "fra"]) on the draft before uploading. [] restores the English default; settings affect subsequent imports, never published cells. Read overmind://dataset-upload for supported codes. Completed pages are checkpointed per project for 30 days; an explicit import resume reuses matching pages, including after worker loss. Inspect pages_reused, recipe_sha256 and actual languages in source extraction metadata. PDF pages missed entirely by the primary native parser are checked with PDFium before OCR. Inspect source native_text_recovery for recovered pages, engine, row/character counts and control_characters. Native text can contain font-encoding artifacts; do not equate extraction success with visual or semantic correctness. The platform preserves these values rather than guessing replacements. Landing stops at a readable source; it does not schedule preparation.

When the user asks to prepare data in Overmind, local artifacts alone are not completion. Publish and inspect the requested dataset, or report an explicit blocker. Use start_dataset's returned upload argv, replacing only documented placeholders. Carry returned identifiers forward instead of retyping them. Inspect the named capability's contract and examples before choosing target semantics; link it explicitly. An entity-matching label does not establish a KYC approval label. Keep unknown meanings unknown. Preserve review candidates and declared groups, assess shortcut features and split contamination, and retain meaningful preparation stages as separate steps rather than only formatting.

MCP and local byte transfer authenticate separately. Upload/export use explicit or repository credentials first, then a saved account connection at $XDG_CONFIG_HOME/overmind/connection.toml (default ~/.config/overmind/connection.toml). It contains api-key and base-url, has permissions 0600, and refuses to send its key to a different selected API. Use the same authorized account and deployment as MCP. Never print credentials. Missing credentials or a transfer failure are connection errors, not reasons to open the Console. Browser navigation requires an explicit user request.

Run overmind connection check --project-id PROJECT --json in the actual coding environment before reporting readiness. Upload performs this check automatically. MCP access does not imply sandbox-local network access. For a confirmed network_permission_denied, request the host's scoped permission for the same command; preserve the file, project and request key. Do not disable the sandbox, ask the user to supply the file again, or infer permission denial from every connection failure.

Uploads bind SHA-256, byte count and destination recipe to a durable transfer. Repeating the identical command resumes it, even after a lost publication reply. --request-key selects an explicit identity; changing its bound inputs conflicts. Inspect get_job(kind=dataset_transfer) for measured bytes and publication; get_job(kind=dataset_run) reports subsequent landing. A pending dispatch is not completed landing. Dataset-run progress.admission and dataset-operation landing_admission show fair-queue waiting, admission, execution and terminal state with measured timestamps. A waiting import consumes no execution attempt; follow polling guidance rather than resubmitting. One project's imports execute serially so another project can use available workers. Failed or lost started attempts retain their source and require explicit recovery through resume_dataset_import; no automatic replay is implied. For a JSON wrapper use --json-rows-field FIELD after inspecting the file's structure. Original wrapper bytes remain retained; the selected field is recorded as extraction provenance. The CLI marks a saved upload state as state_scope=at_publication; it is not current landing progress. Use get_job, or --wait for an observed terminal state.