billy-lyc/neuropilot/dcm2bids-skill

DICOM → BIDS Conversion Skill

Converts raw neuroimaging DICOM datasets into BIDS-compliant format using dcm2niix and dcm2bids command-line tools through a structured, step-by-step workflow.

This skill:

• Analyzes dataset structure
• Detects raw DICOM by DICM magic bytes even when the .dcm extension is missing
• Detects and extracts compressed DICOM archives (.zip, .tgz, .tar.gz)
• Identifies multi-session datasets and maps folders to BIDS ses-<label>
• Classifies imaging series via DICOM metadata
• Generates a valid dcm2bids configuration
• Performs conversion using the dcm2bids CLI
• Validates outputs and reports results

Fast Path — target ≤10 minutes (DEFAULT, read first)

This skill must feel fast. From invocation to a submitted/started conversion should take ≤10 minutes. The #1 thing that makes it slow is stalling on many separate confirmation prompts and dumping huge output. Do not do either. Follow this exactly:

1. Runtime setup + check tools (below + Step 1) — one command each, no prose.
2. Bounded inspection (Step 2) — ONE quick scan. Use head, wc -l, and

find -maxdepth. NEVER print a full directory tree, every subject, or every log file. A few representative lines + counts is enough.

1. Detect prior work and REUSE it. If the output dir already contains a

BIDS dataset, a code/dcm2bids_config.json, or an sbatch script, read them ONCE and reuse — do not re-derive a config from scratch or re-investigate what already exists. Only flag a material discrepancy (e.g. a modality the existing config drops).

1. Cohort-wide series scan + one helper run (Step 3). Two cheap parts:

(a) Series union across ALL subjects — build the set of every distinct series in the cohort cheaply: from series-folder names when the layout has them (find <input> -mindepth 2 -maxdepth 2 -type d | sed 's#.*/##' | sort -u), or one DICOM header per series otherwise. This reads names/one header, NOT a conversion — it is near-instant even for hundreds of subjects, so it does NOT slow the run. (b) One helper run — run dcm2bids_helper on ONE representative subject/session for the detailed sidecar fields (SeriesDescription, Modality, MRAcquisitionType, RepetitionTime). The config must cover the union from (a), not just the one subject in (b): a modality that exists in only some subjects (DWI/ASL/fmap are common examples) must still get a rule, or it is silently dropped for the whole cohort. Step 7's drop-check is the backstop.

1. ASK EVERYTHING ONCE. Collect every open decision into a SINGLE batched

AskUserQuestion call (use its multi-question form). Never ask these one at a time across multiple turns. The batch covers, as applicable:

• input path / output path (if not already given)
• subject scope (test one subject vs. full batch)
• subject-label confirmation (e.g. 002_S_1155 → 002S1155)
• session labeling (sequential 01,02… vs. date string)
• modalities to include
• execution mode (direct vs. `sbatch`) — ALWAYS ASK, NO EXCEPTIONS
• PHI / --anonymize (skip if user already said data is de-identified)

Present the proposed SeriesDescription → datatype/suffix mapping in the SAME message so the user confirms the mapping and answers in one round-trip.

HARD RULE — execution mode is mandatory. You MUST ask whether to run directly on the server or submit a SLURM job (`sbatch`) on EVERY run, even for a single subject, even for a quick test, even if SLURM seems obvious. Never choose it silently and never skip it. The only allowed exception is when sbatch is not on PATH — then say so and fall back to direct execution. This question is non-negotiable; batch it into step 5, but never drop it.

1. Write config + submit. Reuse the existing array/sbatch template if there

is one. Submit, print the job ID, stop.

Do NOT block on intermediate "is this okay?" checks beyond the single batch in step 5. If a default is obvious and reversible, take it and say so. The detailed per-step reference files below are the source of truth for HOW each command runs — but obey the batching and no-dump rules above, which override any "confirm with the user" instruction phrased as a separate prompt in those files.

Runtime setup (REQUIRED — do this first)

dcm2bids, dcm2bids_helper, and dcm2niix must be on PATH before any conversion command. Resolve them in this order:

• If DCM2BIDS_ENV_BIN is set, it wins — use it and do not install anything:

export PATH="$DCM2BIDS_ENV_BIN:$PATH" (On a shared cluster an admin may have staged a prebuilt environment; point DCM2BIDS_ENV_BIN at its bin/ rather than hard-coding a path here.)

• Otherwise activate the environment created during setup:

conda activate dcm2bids-skill

• If neither exists, the environment has not been set up yet — see

ENVIRONMENT.md (one-command conda route: conda env create -f environment.yml).

Verify with dcm2bids --version before proceeding. If it fails, stop and fix the environment rather than working around it.

Dataset Inspection Requirements

Before attempting conversion, always inspect the dataset structure.

Datasets may contain:

• raw DICOM files (.dcm, or extensionless files with DICM magic bytes

at offset 128)

• compressed DICOM archives (.zip, .tgz, .tar.gz)
• previously converted NIfTI files (.nii, .nii.gz) — leave untouched
• segmentation / derived files (.nrrd, .gipl, .mgz) — leave untouched
• mixed or partially processed data

If compressed DICOM archives are detected:

1. Create a temporary extraction directory in the user's writable workspace

($TMPDIR on Linux/Mac, $env:TEMP on Windows). Never extract into the original dataset directory.

1. Extract only the required subject/session data.
2. Never modify the original dataset.
3. Run dcm2niix and dcm2bids on the extracted working copy.

If files have no .dcm extension, use the magic-bytes probe in Step 2 before concluding the directory is empty of DICOM.

If no usable DICOM data is detected, stop and report the issue clearly instead of fabricating a conversion workflow.

Workflow Overview

Follow these reference files in order. Each step must be completed before the next. Do not skip steps.

1. references/01_check_tools.md — verify required CLI tools are installed
2. references/02_analyze_dataset_structure.md — inspect dataset layout,

probe for extensionless DICOM, identify subject/session folders

1. references/02b_handle_archives.md — extract compressed archives (if any)
2. references/03_read_dicom_metadata.md — extract metadata via dcm2bids_helper
3. references/04_classify_series.md — map series to BIDS modalities
4. references/05_write_config.md — write dcm2bids configuration JSON
5. references/06_run_dcm2bids.md — **ask the user the final question: run

directly on the server, or submit a SLURM job (sbatch)** (Step 6.0), then run dcm2bids (single/multi subject, single/multi session) with per-subject failure tolerance, populate participants.tsv, persist the config to code/

1. references/07_validate_and_report.md — run bids-validator, preserve

dcm2bids logs to code/dcm2bids_logs/, clean up working artifacts, write `conversion_report.md` (MANDATORY — see "Mandatory Closing Step" below), and surface the summary

1. Surface the results visually (always do this last). Produce the interactive

output and open it in the panel so the user can browse it — never leave them with only raw files:

• Generate the interactive BIDS-tree HTML with this skill's bundled

generate_bids_viewer.py (it only reads the already-converted BIDS dir to build an HTML tree — it is NOT a conversion or classifier tool, so it is allowed under the Execution Rules): python3 <this-skill-dir>/generate_bids_viewer.py <output_dir> --output <output_dir>/bids_tree.html

• Then call hpc_show_output(connection_id=<id>, directory=<output_dir>). The

output panel lists both artifacts: clicking `bids_tree.html` opens the interactive BIDS structure, and clicking `conversion_report.md` shows the report rendered as formatted HTML by the panel's built-in markdown viewer (the user never sees raw .md).

Mandatory Closing Step — conversion_report.md (NON-NEGOTIABLE)

If the conversion succeeds, you MUST generate `conversion_report.md`. No exceptions. A run that converts data but does not leave behind <output_dir>/conversion_report.md is an incomplete run, not a successful one.

"Conversion succeeds" means at least one subject/session produced BIDS output (NIfTI + sidecars under sub-<label>/). The moment that is true, the report is required — even for a single-subject test run, even for an sbatch batch, even if bids-validator reports warnings.

Rules:

• The report is not produced automatically by any CLI tool. dcm2bids,

dcm2niix, and bids-validator never write it. You write it, by hand, following Step 7d in references/07_validate_and_report.md. If you skip that step, the file does not exist.

• Write it to the exact path <output_dir>/conversion_report.md and add it to

.bidsignore (per Step 7).

• For an sbatch submission, the report covers the submission (job ID, what

was submitted, where outputs will land). After the job finishes you still owe a report reflecting actual results — generating/updating conversion_report.md is part of "done", and is the agent's responsibility, not the scheduler's.

• Treat writing this file as the literal definition of finishing. Do not tell

the user the conversion is complete until conversion_report.md exists on disk at the path above. Before declaring success, verify the file is present (e.g. ls -l <output_dir>/conversion_report.md).

• The only case where the report may be omitted is a run that produced no

BIDS output at all (total failure) — and even then, report the failure to the user clearly.

Performance — Stay On-Rails (do not rabbit-hole)

Conversions stall when the assistant hand-rolls the pipeline instead of following the steps. To keep runs fast:

• Bounded inspection. A quick scan is enough to design the mapping: list the

dataset tree (Step 2), then run dcm2bids_helper on ONE representative subject/session (Step 3). Do NOT exhaustively read headers from every subject before starting.

• Confirm the mapping, then run. After Steps 4–5, show the proposed

SeriesDescription → datatype/suffix mapping (the config) to the user for a quick confirmation. That is the moment to catch a wrong mapping — not after a full batch convert.

• Do not debug tool internals. Never read or step through dcm2bids /

dcm2niix source code (e.g. sidecar.py, searchDcmTagEntity). If a command fails, show the exact command and its error, try at most one or two targeted fixes, then surface the problem and ask the user.

• **One config, reused for the batch — but built from the whole-cohort series

union.** Build ONE config and reuse it for every subject; do NOT iterate the config subject-by-subject. Derive the series list from the cheap cohort-wide scan (Step 3a / Step 4a: folder names or one header per series across ALL subjects), NOT from a single representative subject — otherwise modalities present in only some subjects (DWI/ASL/fmap) are silently dropped cohort-wide. The cohort scan is name/header-only, so it stays fast. Per-subject failure tolerance is handled in Step 6; the Step 7 drop-check is the final backstop.

Always ask for input/output paths and execution mode before running (see Output Rules and Execution Mode). Never silently pick paths or grind without surfacing progress.

Privacy and PHI