timscheuerai/content-vault/shorts-edit

shorts-edit

The orchestrator. Raw takes in, postable vertical shorts out.

Built and hardened on a real session: 14 OBS recordings, 57 minutes of raw → 21 minutes of cut. Every rule below exists because a defect got through review looking fine.

The prime directive

Nothing is done until `qa.py` passes. Not "looks good" — passes.

Every single bug in this pipeline's history was invisible to eyeballs and obvious to a measurement. A caption verifier that sampled long words reported perfect while short words were silently wrong. A tier grade was revealed 5.7s before it was spoken and every frame looked fine in isolation. Audio "sounded weird" and the cause was a filter silently changing mode. Do not trust your eyes on this material. Measure.

Subskills

Read the one you need. Read shorts-qa always.

Project layout

<project>/
  raw/     source recordings (untouched)
  clips/   <name>_final.mp4   <- deliverables
  edit/    everything derived
    transcripts/       cached word-level ASR — NEVER re-transcribe
    edl_parts_tight/   the cut decisions — SOURCE OF TRUTH
    motion.json        the graphics plan (cue-anchored)
    word_fixes.json    ASR corrections + caption inserts
    caption_work/      per-frame PNG sequence + schedule.json

Scripts find the project via $SHORTS_PROJECT, --project <dir>, or by walking up from the cwd for raw/.

SETUP

brew install ffmpeg                 # every stage shells out to it
pip install pillow numpy            # the renderers
 
# transcription runs through the vendored video-use helpers
export ELEVENLABS_API_KEY=...

Then point two shell vars at this vault so the pipeline below copy-pastes:

V=~/.claude/skills/content-vault/skills    # wherever you cloned content-vault
S=$V/shorts-edit/scripts

Optional, all with sane fallbacks — see scripts/assets.py:

The pipeline

export SHORTS_PROJECT="/path/to/project"
 
# 1. transcribe (word-level, CACHED)
python $V/video-use/helpers/transcribe_batch.py "$SHORTS_PROJECT/raw" \
       --edit-dir "$SHORTS_PROJECT/edit" --workers 4
python $V/video-use/helpers/pack_transcripts.py --edit-dir "$SHORTS_PROJECT/edit"
python $S/words_dump.py
 
# 2. pick takes   -> edit/edl_parts/*.json     (see shorts-cut)
# 3. tighten dead air inside kept takes
python $S/tighten_edl.py                        # -> edit/edl_parts_tight/
 
# 4. plan graphics (see shorts-motion)
python $S/motion_scout.py "<clip>"              # find the visualizable moments
#   -> hand-write edit/motion.json, CUE-ANCHORED
 
# 5. build — cut + captions + audio + graphics + polish, ONE encode
python $S/build.py "<clip>" --polish
 
# 6. THE GATE — nothing ships until this passes
python $S/qa.py "<clip>"

Hard rules (violating any of these produces a silent failure)

1. ONE video encode. Trim → concat → zoom → overlay → encode, all in one filtergraph.

Two encodes softened a 246 kbps source from SSIM 0.998 to 0.977. Never re-encode a finished clip to add something to it.

1. ASR word END times are fiction. They pad into the following silence and can be

phantom words over dead-silent audio. Word STARTs are reliable. All silence reasoning comes from the audio (`silencedetect`/RMS), never the transcript.

1. Never let ffmpeg derive caption/graphic timing from durations. The concat demuxer

re-times to CFR and rounds boundaries as it pleases. Use one PNG per output frame.

1. Never use `loudnorm` for gain. See shorts-audio. It silently pumps.
2. Graphics are cue-anchored, never hand-timed. See shorts-motion.
3. Fades only at real cut edges. A mid-beat framing change is not a cut — the audio

runs straight through it. A 30ms fade there dips the voice audibly.

1. `concat` wants inputs interleaved v0,a0,v1,a1,… — not all video then all audio.

The two are identical for one segment, so a single-segment test clip hides the bug.

1. PIL's ImageDraw with a partial-alpha fill OVERWRITES pixels, it does not composite.

Draw at full opacity on a layer, then fade the layer. Otherwise a fading element punches a transparent hole through whatever is beneath it.

What shipped

Fix the capture, not the edit

The source was 360×640, 246 kbps, −43 LUFS. No post-processing recovers detail that was never recorded. Raising the OBS canvas and input gain improves every future edit for free.