tsiendragon/qwen-image-finetune/qwen-image-edit-aipc-finetune-02-dataset

Step 2 — Dataset Preparation

Series position: Step 2 of 8.

Next step: when dataset_validate.py exits 0 and the test set is set aside, proceed to skill qwen-image-edit-aipc-finetune-03-env-setup.

The qwen-image-finetune framework (qflux.data.dataset.ImageDataset) supports three dataset formats. Pick whichever fits the user's workflow; all three are validated by the script in §2.7.

2.1 Format options

Where the data comes from (HuggingFace Hub, ModelScope, locally created, etc.) is a matter of source, not format. Download the dataset to a local path first, then point the training config at the local path in the matching format above.

Detailed schemas live in src/qflux/data/dataset.py (ImageDataset.__init__ docstring). The Step 4 recommender emits a data: block matching whichever format the user has.

2.2 Image directory layout

dataset_root/
├── training_images/
│   ├── sampleA.png              # target image — the desired OUTPUT after the edit
│   └── sampleA.txt              # required: prompt text (the edit instruction)
└── control_images/
    ├── sampleA.png              # main control — INPUT condition (e.g. background scene)
    ├── sampleA_control_1.png    # optional: 2nd control (e.g. character reference image)
    └── sampleA_mask.png         # optional: mask of the edited region (character silhouette)

Naming rules (from qflux.data.dataset.ImageDataset documentation):

• Extra control images: append _control_1, _control_2, … to the base name
• Mask: <base>_mask.png in either control_images/ or training_images/
• Prompt: .txt file with the same base name; if present in both dirs, training_images/ wins

Alternative directory names accepted: images / target_images / target for targets; control / condition_images / controls for controls.

Multi-control example (character composition task): sampleA.png = background scene, sampleA_control_1.png = character on white background, sampleA_mask.png = character silhouette in the final composition, prompt = "Add the character to the image". This is the pattern used by the TsienDragon/character-composition reference dataset.

Mask usage: when a mask is present, edit_mask_loss applies higher training weight to the masked region, focusing the LoRA on the area that changed between control and target. Useful when only part of the image is edited.

2.3 Sample size guidance

Quality beats quantity. Adding low-quality or inconsistent pairs actively harms the LoRA — they introduce noise the model cannot learn a clean pattern from. ¹ ²

Single-task LoRA preferred. If you want to train multiple distinct edits (e.g., style + object replacement), train separate LoRAs rather than one mixed dataset. Multi-task LoRA often causes the tasks to interfere with each other. ²

The validator (§2.7) enforces the floor of 5 samples and warns below 30. The framework caches embeddings per-sample (Step 7), so larger datasets cost more upfront cache time but no extra per-step cost during fit.

¹ FlyMyAI LoRA Trainer, https://github.com/FlyMyAI/flymyai-lora-trainer, Aug 2025. Uses a different training framework; cited for model-level image quality guidance applicable to Qwen-Image-Edit. ² HuggingFace Forums: "Question about lora fine tune qwen-image-edit" (John6666, Nov 2025), https://discuss.huggingface.co/t/question-about-lora-fine-tune-qwen-image-edit/170633. Qwen-Image-Edit-specific community guidance; cited for dataset size ranges, quality advice, and single-task LoRA recommendation. ³ とりにく, "vast.AIでQwen image Edit 2509のLoRA学習", https://note.com/tori29umai/n/n256f30d51669, Sept 2025. Uses Musubi Tuner (a separate training framework); dataset construction and spatial transformation advice is model-level and framework-independent.

2.4 Control image and target image — the edit pair

Qwen-Image-Edit is an image editing model, not an image generation model. Every training sample teaches the model one specific edit:

The key difference from caption-based LoRA training (e.g. for image generation): the prompt describes what to do, not what the result looks like.

Multi-control paradigm — the framework natively supports multiple control images per sample, which is useful when the edit requires more than one reference:

• Main control (sampleA.png): the primary input scene or reference
• Additional controls (sampleA_control_1.png, _control_2.png, …): supplementary

references (e.g. a character sheet, a style reference)

• Mask (sampleA_mask.png): region of interest — tells the model which part of the image the

edit targets

Verified example (character-composition task): the TsienDragon/character-composition dataset is one verified instance of this setup — it uses two controls (a background image and a character-on-white-background image) plus a mask of the character's silhouette, with a fixed prompt "Add the character to the image". The target is the character correctly composited into the background. User-created datasets following the same format work equally well.

General task examples (for reference, not exhaustive):

• Viewpoint change: control = source angle; target = desired angle; prompt = describe the viewpoint transformation ³
• Object addition: control = scene; target = scene with object added; prompt = "Add [object] to the image"
• Style change: control = original; target = restyled; prompt = describe the style change

Important: control and target must share the same subject/context. Unrelated images in a pair prevent the model from learning a coherent edit mapping.

Practical tip: synthetic or rendered control images (e.g. 3D, game engine) produce very consistent results because they eliminate photographic variation that the model might otherwise try to replicate. ³

³ とりにく, "vast.AIでQwen image Edit 2509のLoRA学習", https://note.com/tori29umai/n/n256f30d51669, Sept 2025. Uses Musubi Tuner (a separate training framework); dataset construction advice is model-level and framework-independent.

2.5 Image quality guidelines

• Resolution: source images should be at least as large as your `target_size` in each

dimension. The training preprocessor crops/resizes source images down to target_size — if the source is smaller, it gets upscaled first (quality loss).

• Source resolution larger than target_size is always fine; the crop just has more to choose from.
• A practical floor: source images should be at least as large as your chosen target_size in

both H and W. If your images are already at or above the target_size the recommender emits (matched to your dataset's shape), no additional action is needed. (Recommended range from `docs/guide/data-preparation.md`: 512×512 to 1024×1024 — appropriate for moderate `target_size` configurations used with this series.)

• Aspect ratio — set `target_size`'s shape to match your dataset. target_size is

[height, width]. With the default process_type: center_crop, the preprocessor scales each image to fill target_size and crops whatever does not fit — so if the target's proportions differ from your images, part of every image is cropped away. Match the shape (ratio) to your data: a square dataset needs a square (H = W) target; a landscape dataset a wide (W > H) target; a portrait dataset a tall (H > W) target. To keep the whole frame instead of cropping, use process_type: center_padding (pads with borders rather than cropping).

• Edge lengths are a memory question — and `target_size` is not the only input. The DiT

processes target, all control images, and the text prompt together in joint attention; the total token count across all of them drives memory. Each image's token count scales with its pixel area (larger target_size or controls_size = more tokens = more memory). The text prompt token count is roughly fixed and not affected by image size settings. The number of control images comes from your dataset, not the config: the main control always counts, and additional _control_N images add to it only when your samples provide them (controls_size merely gives the size to use for each — its 2nd entry applies only if a 2nd control exists). More or larger control images cost more memory.

• Pick edge lengths for your own dataset by measuring, not from a fixed number. The Step 4

§4.5 resolution tiers are the recommender's starting points; because the total cost depends on your control count and sizes too, keep the shape matched to your data, start conservative, and confirm headroom with monitor_xpu_memory.py (Step 7) before increasing. Both dimensions must be multiples of 16. Changing target_size or controls_size requires rebuilding the cache (Step 7).

• Format: JPG, JPEG, PNG, WebP — all supported by the framework. RGB (3-channel) required.
• Consistency within dataset: maintain consistent image quality, lighting style, and

subject framing across pairs. Inconsistency is a common cause of blurry or incoherent model outputs. ¹

• Quality over quantity: more low-quality pairs actively harm the LoRA — they introduce

noise the model cannot learn a clean pattern from. ¹

• Avoid: watermarks or text overlays in control/target images; heavy JPEG artefacts;

mixing very different visual styles (e.g., anime and photorealistic) in the same dataset without a compelling reason.

¹ FlyMyAI LoRA Trainer, https://github.com/FlyMyAI/flymyai-lora-trainer, Aug 2025. Uses a different training framework; cited for model-level image quality guidance applicable to Qwen-Image-Edit.

2.6 Prompt strategy

The framework expects prompts to be descriptive editing instructions — text that tells the model what to do, not what the result looks like. (From `docs/guide/data-preparation.md`: "Descriptive editing instructions"; recommended length 10–200 words)