multigent/multigent/asset-director

Asset Director — Explainer Pipeline

When to Use

You are the Asset Producer for a generated explainer video. You have a scene_plan with required assets and a script with narration text. Your job is to generate every asset needed: narration audio, images, diagrams, code snippets, and background music. Every file must exist on disk before you finish.

This is where plans become real files. A missing or low-quality asset will torpedo the final video.

Animation authoring — which runtime

Before authoring any animated Remotion component for this pipeline, read `skills/meta/animation-runtime-selector.md`. It's the routing authority for deciding between Remotion primitives and GSAP plugins.

Quick routing for common explainer needs:

The keep-it-simple bias: if Remotion primitives solve a scene in ≤ 20 lines, use them. Only pull in GSAP when the plugin genuinely earns its bundle weight.

Prerequisites

Process

Step 1: Inventory Required Assets

Walk every scene in the scene plan. For each required_assets entry, create an asset task:

Asset Task:
  scene_id: scene-3
  type: diagram
  description: "Mermaid flowchart: query -> encode -> search -> rank -> return"
  source: generate
  tool: diagram_gen
  estimated_cost: $0.00

Also create tasks for:

• Narration audio — one per script section (use tts_selector or a concrete TTS provider)
• Background music — one track for the whole video (use music_gen or select from library)
• Sound effects — per playbook's sfx_style (optional, use music_gen or stock)

Step 2: Check Budget

Before generating anything:

1. Sum all estimated costs from the asset tasks
2. Compare against the cost tracker's remaining budget
3. If over budget:

• Switch expensive tools to cheaper alternatives (use tts_selector with preferred_provider to route to cheaper TTS; use image_selector to route to cheaper image providers)
• Reduce image count (combine similar scenes)
• Skip optional assets (SFX, B-roll)

1. Get cost approval via cost tracker before proceeding

Step 2b: Sample Preview (Prevents Wasted Spend)

Before batch-generating assets, produce one sample of each expensive asset type and present them to the user for approval:

1. TTS sample: Generate narration for script.voice_performance.sample_section_id when present; otherwise pick the section with the most demanding delivery. Play it for the user. Confirm voice, pace, pauses, emphasis, and tone are acceptable before generating the rest.
2. Image sample: Generate one image for the most representative scene. Show it to the user. Confirm the style, quality, and prompt approach before batch-generating all images.
3. Music sample (if using music_gen): Generate one short clip. Confirm mood and energy before committing.

If the user rejects a sample:

• Adjust the parameters (voice, prompt style, provider) and regenerate the sample.
• Do not batch-generate until the sample is approved.
• Max 3 sample iterations per asset type before escalating to the user for a decision.

This step typically costs $0.03–0.08 total and prevents $1–3 of wasted generation.

Step 3: Generate Narration

For each script section:

1. Extract the narration text
2. Read script.voice_performance and section delivery_cues
3. Use delivery_cues.provider_text when present; otherwise transform the section text with purposeful punctuation and break tags only when the selected provider supports them
4. Apply speaker directions from the script (pace, emphasis, emotion)
5. Apply the playbook's audio.voice_style
6. Map cues to provider parameters:

• OpenAI: instructions only with model: "gpt-4o-mini-tts"; use response_format for output format
• Google TTS: input_type: "ssml" when using <break> tags, plus speaking_rate in 0.25..2.0 and pitch in -20..20
• ElevenLabs: stability, similarity_boost, style, speed, and use_speaker_boost

1. Generate using tts_selector — it auto-routes to the best available TTS provider based on user preference and availability. Check the registry's best_for fields to understand each provider's strengths.
2. Record the applied voice_performance metadata on each narration asset
3. Verify the audio file exists and duration matches expected timing (±15%)

Pronunciation guide: If the script contains technical terms, jargon, or names with non-obvious pronunciation, include a pronunciation map in the TTS request.

Flat voice failure: If the approved voice sounds monotone, robotic, rushed, or ignores intended pauses, do not batch the remaining sections. Revise the voice_performance plan or provider parameters and regenerate the sample.

Step 4: Generate Visual Assets

Process asset tasks grouped by tool for efficiency:

Images (`image_selector`):

1. Build the prompt from the scene's actual purpose:

• scene-specific shot/lighting/texture cues from shot_language, shot_intent, and texture_keywords
• an adapted visual anchor from the playbook or custom identity
• the concrete subject/action/environment

Use lib/shot_prompt_builder.py when helpful.

1. Add negative prompt from playbook
2. Include consistency anchors (same character/world/palette family), but do NOT reuse the exact same phrasing for every image
3. Generate and verify the file exists
4. If the result doesn't match expectations, refine the prompt and regenerate (max 2 retries)

Diagrams (`diagram_gen`):

1. Convert the scene description into valid Mermaid syntax
2. Apply playbook's asset_generation.diagram_style
3. Generate SVG/PNG
4. Verify all nodes and edges are present

Code snippets (`code_snippet`):

1. Extract language and code from the scene description
2. Apply syntax highlighting theme from playbook's overlay styles
3. Generate highlighted image or Remotion-compatible data

Step 5: Generate Music

1. Read playbook's audio.music_mood and audio.music_volume
2. Check the music decision from proposal_packet.production_plan.music_source (set by the Proposal Director)
3. Source the background track in this priority order:

• User-selected library track: If the proposal specified a track from music_library/, copy it to projects/<project>/assets/music/background_music.mp3
• User music library (`music_library/`): If the folder exists and has tracks, pick the best match for the playbook's audio.music_mood. List candidates by filename and let the EP decide.
• Music generation API: Use music_gen (ElevenLabs) or suno_music if available. Check status via registry first — if the tool is unavailable or quota-exhausted, skip immediately (do NOT attempt and fail silently).
• No music available: Log this clearly in the asset manifest as "music_status": "unavailable" with the reason. Do NOT silently produce a video without music — the EP and user should know.

1. Duration should be at least as long as total video duration. If shorter, it can be looped by the compose stage.
2. Verify the audio file exists at projects/<project>/assets/music/background_music.mp3

Critical: If music generation fails or is unavailable, report it immediately in the asset manifest — do not defer the problem to the compose stage.

Step 6: Build Asset Manifest

Assemble all generated assets into the manifest:

{
  "version": "1.0",
  "assets": [
    {
      "id": "narration-s1",
      "type": "audio",
      "subtype": "narration",
      "path": "assets/narration/s1.mp3",
      "source_tool": "tts_selector",
      "scene_id": "scene-1",
      "duration_seconds": 8.2,
      "cost_usd": 0.003
    },
    {
      "id": "img-scene-3",
      "type": "image",
      "path": "assets/images/scene-3-diagram.png",
      "source_tool": "diagram_gen",
      "scene_id": "scene-3",
      "cost_usd": 0.00
    },
    {
      "id": "music-bg",
      "type": "audio",
      "subtype": "music",
      "path": "assets/music/background.mp3",
      "source_tool": "music_gen",
      "duration_seconds": 62,
      "cost_usd": 0.05
    }
  ],
  "total_cost_usd": 0.053,
  "generation_summary": {
    "narration_sections": 5,
    "images_generated": 8,
    "diagrams_generated": 2,
    "music_tracks": 1
  }
}

Pre/Post Self-Review for Generation Prompts

Before sending a prompt to any generation tool — image_selector, diagram_gen, video_selector, even code_snippet styling prompts — run a three-step self-review modeled on the CHAI oversight loop ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2). Cost is small (no extra tool calls); benefit is large (avoids wasted generations). This applies just as strongly to diagram_gen Mermaid prompts and image_selector illustration prompts as it does to video generation — bad explainer visuals fail in the same way: missing subject, missing framing, vague "make it look educational."

Step 1 — Pre-caption pass. Write the prompt the way you'd write it today. Do not over-edit; aim for a complete first draft.

Step 2 — Critique pass. Score the draft against the 5-aspect checklist (Subject / Subject Motion / Scene / Spatial Framing / Camera). For each aspect:

- Is it specified? If not, is the omission deliberate (e.g., "Camera N/A — Remotion native scene", "no subject motion — static diagram") or accidental?

- Are confusable terms disambiguated? (dolly vs zoom, pan vs truck, bird's-eye vs aerial, fisheye vs barrel, full shot vs close-up; for diagrams: flowchart vs sequence vs state diagram, top-down vs left-right)

- Are emotional adjectives ("clean", "professional", "modern") replaced with their visual causes (sans-serif typography, generous whitespace, monochromatic palette with one accent)?

- For multi-shot prompts: is identity anchored verbatim across shots? For image_selector prompts that recur (a character or world appearing in multiple scenes), are consistency anchors specified verbatim?

Step 3 — Post-caption pass. Rewrite filling the missing aspects, fixing confusable terms, and replacing subjective language. The post-caption is what gets sent to the generation tool.

Log the (pre, critique, post) triplet in the asset metadata for traceability. This mirrors the CHAI workflow and creates a record the reviewer can audit.

Step 7: Verify All Assets