mckruz/comfyui-expert/comfyui-video-production

1. Keyframe Generation Phase
   - Generate consistent keyframes with IP-Adapter/LoRA
   - Validate face consistency, lighting, pose progression
   - Save to organized directory structure
   - Auto-retry failed generations
 
2. I2V Animation Phase
   - Queue each keyframe to I2V model (Wan 2.2, LTX-2, AnimateDiff)
   - Monitor progress via ComfyUI API
   - Validate each clip (resolution, fps, duration)
   - Auto-retry with different seeds if failed
 
3. Concatenation Phase
   - Pre-flight validation (ensure all clips match)
   - Apply transition effects (crossfade, motion blur)
   - FFmpeg encoding with proper codec
   - Export final video with metadata
 
4. Quality Assurance
   - Face consistency check across clips
   - Color grading consistency
   - Audio sync validation (if applicable)
   - Generate QA report

1. Image Discovery
   - Scan directory for source images
   - Validate image specs (resolution, format)
   - Generate processing manifest
 
2. Parallel I2V Queue
   - Queue all images to ComfyUI with appropriate prompts
   - Stagger submissions to avoid overload
   - Monitor queue depth and ETA
 
3. Progressive Validation
   - Check each completed video immediately
   - Flag issues (wrong resolution, fps, corruption)
   - Auto-retry flagged videos
 
4. Export & Organize
   - Move validated videos to output directory
   - Generate index with metadata
   - Create contact sheet (thumbnail preview grid)

1. Clip Validation
   - Verify all clips exist and are readable
   - Check resolution, fps, codec consistency
   - Report mismatches with fix suggestions
 
2. Transition Planning
   - Detect scene changes (cut detection)
   - Recommend transition types (crossfade, zoom, pan)
   - Calculate transition timing
 
3. FFmpeg Pipeline
   - Apply transitions between clips
   - Re-encode with consistent settings
   - Preserve quality (high bitrate, proper codec)
 
4. Audio Handling
   - Extract audio from clips (if present)
   - Crossfade audio at transitions
   - Sync to final video timeline

# Auto-detected issues:
- Queue stalled (no progress for 5+ minutes)
- Memory leak (VRAM usage climbing)
- Process crashed (connection refused)
- API unresponsive (timeout on /queue endpoint)
- Disk full (output directory at capacity)

1. Soft Recovery (no restart)
   - Clear stuck queue items
   - Force garbage collection
   - Unload models from VRAM
2. Hard Recovery (restart required)
   - Save current queue state
   - Kill ComfyUI process gracefully
   - Wait for port release
   - Restart with same config
   - Restore queue from saved state
3. Emergency Fallback
   - Switch to backup ComfyUI instance
   - Redirect queue to instance on different port
   - Continue processing without data loss

# Run multiple ComfyUI instances for parallel processing
Instance 1: localhost:8188 (primary - I2V generation)
Instance 2: localhost:8189 (secondary - upscaling/post-processing)
Instance 3: localhost:8190 (backup - standby for failover)
 
# Load balancing strategy:
- Round-robin for equal workloads
- Priority-based for mixed tasks
- Failover for crashed instances

✓ Check ComfyUI is running and responsive
✓ Verify models are loaded (UNET, VAE, CLIP)
✓ Confirm output directory has sufficient space
✓ Validate source images exist and are readable
✓ Check prompts are non-empty and formatted correctly
✓ Verify workflow JSON is valid

✓ Video file exists and is non-zero size
✓ Resolution matches expected (e.g., 768x1024)
✓ FPS matches expected (e.g., 16 or 25)
✓ Duration matches expected (e.g., 3-5 seconds)
✓ Codec is compatible (h264, h265)
✓ No corruption (can read all frames)
✓ Face consistency score >0.85 (if character video)
✓ Color histogram within expected range

Metrics tracked:
- Face embedding distance (identity consistency)
- Optical flow magnitude (motion smoothness)
- Frame PSNR/SSIM (interpolation quality)
- Color histogram deviation (lighting consistency)
- Audio sync offset (if audio present)

1. Seed Randomization Retry
   - Failed generation? Try different seed
   - Max 3 attempts per keyframe
   - Track seeds that fail (avoid reuse)
2. Parameter Adjustment Retry
   - CFG too high causing artifacts? Lower it
   - Steps too low causing incompleteness? Increase
   - Resolution too high OOM? Downscale
3. Model Fallback Retry
   - Wan 2.2 14B OOM? Fall back to 1.3B
   - LTX-2 unavailable? Fall back to Wan 2.1
   - AnimateDiff motion broken? Switch motion LoRA
4. Checkpoint Resume
   - Save progress after each successful clip
   - Resume from last successful checkpoint
   - Skip already-generated clips

logs/
├── 2026-02-16_pipeline.log       # Main pipeline log
├── 2026-02-16_comfyui.log        # ComfyUI stdout/stderr
├── 2026-02-16_validation.json    # Validation results
├── 2026-02-16_failures.json      # Failed attempts with reasons
└── 2026-02-16_recovery.json      # Recovery actions taken

project_name/
├── 00_keyframes/                 # Source keyframe images
│   ├── kf01_scene_description.png
│   ├── kf02_scene_description.png
│   └── ...
├── 01_clips/                     # Individual animated clips
│   ├── clip_001_kf01.mp4
│   ├── clip_002_kf02.mp4
│   └── ...
├── 02_validated/                 # Clips that passed validation
│   ├── clip_001_kf01.mp4
│   ├── clip_002_kf02.mp4
│   └── ...
├── 03_transitions/               # Intermediate files for transitions
│   ├── transition_001_002.mp4
│   └── ...
├── 04_final/                     # Final combined video
│   ├── final_video_v1.mp4
│   ├── final_video_v2.mp4        # After revisions
│   └── ...
├── logs/                         # Execution logs
├── metadata/                     # JSON metadata for each asset
└── manifest.json                 # Complete project manifest

# Configuration
project_name = "sage_character_reveal"
keyframes = 5
i2v_model = "wan_2.2_moe"
target_duration = 30  # seconds
fps = 16
# Pipeline execution
1. Generate 5 keyframes (IP-Adapter + LoRA)
   → sage_kf01_over_shoulder.png
   → sage_kf02_turning.png
   → sage_kf03_cardigan_fallen.png
   → sage_kf04_removing_bra.png
   → sage_kf05_topless.png
2. Validate keyframes
   → Face consistency: 0.92 ✓
   → Lighting consistency: 0.88 ✓
   → Pose progression: logical ✓
3. Queue I2V for each keyframe
   → clip_001: 6s @ 16fps (96 frames) ✓
   → clip_002: 6s @ 16fps (96 frames) ✓
   → clip_003: 6s @ 16fps (96 frames) ✓
   → clip_004: 6s @ 16fps (96 frames) ✓
   → clip_005: 6s @ 16fps (96 frames) ✓
4. Apply 0.5s crossfade transitions
   → Total: 30s - 2s (4 transitions × 0.5s) = 28s net
5. Export final video
   → sage_character_reveal_final.mp4 (30s, 768x1024, 16fps)

# Configuration
input_dir = "E:/ComfyUI/input/character_expressions"
i2v_model = "ltx_2"
motion_prompt = "gentle breathing, subtle movement, natural"
batch_size = 4  # Process 4 at a time
# Pipeline execution
1. Scan input directory
   → Found 20 PNG files
2. Queue 4 at a time to ComfyUI
   → Batch 1: expr_001.png → expr_004.png ✓
   → Batch 2: expr_005.png → expr_008.png ✓
   → Batch 3: expr_009.png → expr_012.png ✓
   → Batch 4: expr_013.png → expr_016.png ✓
   → Batch 5: expr_017.png → expr_020.png ✓
3. Validate each output
   → 19/20 passed (expr_011 failed - wrong resolution)
   → Retry expr_011 with corrected settings ✓
4. Export batch
   → 20 validated clips in output/expressions/
   → Generated contact sheet: expressions_preview.png

# Automatically adjust settings based on available resources
if vram_available > 24:
    use_model = "wan_2.2_moe_14b"
    resolution = (832, 1216)
    batch_size = 1
elif vram_available > 12:
    use_model = "wan_2.1_1.3b"
    resolution = (768, 1024)
    batch_size = 2
else:
    use_model = "animatediff_v3"
    resolution = (512, 768)
    batch_size = 4