gtkottman/mortiflix-oss/music

Music

One engine makes all of Mortiflix's music, a song or a video's score. It works in stages; each stage writes a file, the engine checks it, and nothing moves on until the check passes. The notes and the sounds are separate: the score is only music (notes on channels), its MIDI is captured, and only then is an instrument chosen for every channel. So the same MIDI can be played by Strudel or taken into any DAW.

genre ─▶ blueprint ─▶ score ─▶ MIDI ─▶ instruments ─▶ audition ─▶ master
(songs)    the plan    notes    every     a sound per     each sound     Strudel's render, mastered
                       only     channel   channel, from   across its     ─ or ─
                                          its real notes  range          the owner's own, from the MIDI pack

After every command the engine rewrites `music/MUSIC-SHEET.md`: every channel with its range, instrument, level, audition result and why, the story measured against the plan, the master's numbers. It's how the owner (and the next session) sees exactly how the music was made. node $M sheet rewrites it on demand.

M=.claude/skills/music/strudel.mjs

Read craft/genre-research.md before a genre, craft/picture-scoring.md before a video's first blueprint, craft/melody-craft.md before writing the motif, craft/harmony-lock.md before the score, and craft/rich-voicing.md once the lock passes.

0. What the owner set up

$MFX_MUSIC is {"engine": "strudel" | "none", "midi": true | false}. Strudel is at $MFX_STRUDEL, and renders in the Chrome at $MFX_CHROME. If either is missing: mfx needs-you "Music isn't set up: run mortiflix setup music, then resume." Never improvise music any other way, and never use another engine.

Who finishes the music comes from the brief (the music or finish answer):

• "…I'll finish it from the MIDI" (recommended for videos): the studio writes and checks the score, chooses

stand-in instruments so it can be reviewed against the picture, and hands over the MIDI pack; the owner makes the final sound in their own DAW and imports the master (section 9).

• "…rendered by Strudel": the studio finishes it: instruments, audition, render, mix.

1. The genre (songs)

Research the genre into music/genre.json (the format and the method: craft/genre-research.md), then node $M genre until it passes. A song is written from its genre: its tempo range, lineup, beat grids, harmony, hook traits and, above all, its unforgettable elements.

2. Read the drama

Write in music/SPOTTING.md (videos) or at the top of the blueprint (songs):

• The dramatic reading in one sentence, as a transformation: confusion -> recognition -> confidence. For a song,

it's the topic translated into audible behaviour: what the listener feels change from the first bar to the last.

• What the music must not do: compete with the voice, hit every cut, explain what the picture already says

(videos); copy a reference, pile up unrelated ideas, stay at one level (songs).

3. Spot the picture (videos)

Build the spotting map from the real timing: voice/timing.json (every line and word), the animatic's cut and beat list (video/src/timing.ts or its plan), and its length (ffprobe). One row per event that changes meaning:

time    | event                         | significance            | response
0.00    | black, first word             | establish               | pad alone, low register
8.40    | "...for the first time."      | the problem lands       | aftermath: drop out for a bar
31.20   | logo resolves                 | structural: the turn    | direct hit, tonic arrival

Classify each as structural, dramatic, physical or optional, and choose for each: direct hit, anticipation (change just before), aftermath (let the image land, then react) or no hit. Hit the few that change meaning, not every cut.

Map the tempo

Choose the BPM (and offset_sec, where bar 1 starts in the video) so the structural hits fall on bar lines or strong beats: seconds per bar = beats_per_bar × 60 / BPM. Test tempos, don't force awkward bars:

// candidate tempos, scored by how far each hit lands from its nearest beat
for (let bpm = 70; bpm <= 140; bpm += 0.5) { const beat = 60 / bpm; const err = hits.map((t) => Math.abs(((t - off) / beat) - Math.round((t - off) / beat)) * beat); /* keep the smallest max(err) */ }

A minor event may sit between beats. If one tempo can't serve the structural hits, move offset_sec, or use a pickup, a held bar or a section of a different length inside the music, never a tempo that drifts.

4. The blueprint: music/blueprint.json

{
  "title": "a real title", "bpm": 96, "meter": "4/4", "bars": 48, "scale": "D:dorian", "offset_sec": 0.6,
  "dramatic_reading": "confusion -> recognition -> confidence",
  "style": "songs: one dense line of descriptors, under 1000 characters: genre, instruments and their roles, groove, mood",
  "genre_signature": [ { "id": "dusty-breakbeat", "carries": "beat", "how": "half-time break, ghosted snares", "where": "from bar 9" } ],
  "motif": { "rhythm": "...", "contour": "...", "length_bars": 2, "role": "Glass hook", "returns": ["bar 17, an octave up", "bar 41, augmented"] },
  "sections": [
    { "name": "The quote that comes back", "start_bar": 1, "end_bar": 8, "function": "establish: pad and sub only, the motif withheld", "intensity": 2, "picture": "the $5,400 price counts up" }
  ],
  "hits": [ { "t_sec": 31.2, "bar": 13, "beat": 1, "kind": "direct", "what": "logo resolves" } ],
  "chart": [ { "bars": [1, 8], "chords": ["Dm9", "Cmaj7", ["Gm7", "A7sus4"], "Dm9"] } ],
  "roles": [ { "channel": 1, "harmony": "bass", "name": "Sub", "job": "roots, the floor", "instrument_idea": "sine, low-passed" } ],
  "ending": "how it ends and why (a held chord under the end card, or a question left unresolved)"
}

Videos use offset_sec, hits and picture; songs use style and genre_signature (and leave the others out).

Decide in this order: the motif and hook (melody craft) → tempo, meter and scale → the chart → the roles → the story.

• Sections cover bars 1..bars, contiguous. Names are specific and story-bearing, never "Intro/Verse/Chorus/A/B"

(a returning idea is "Return of …"). Each function says what happens: establish, develop, destabilise, withhold, release, reinterpret, resolve, and what rises or is held back.

• `intensity` (0-10) is the energy curve, section by section (a video's follows the picture): the check measures

each section (parts playing, notes per beat, velocity, range) and fails if the order disagrees. Builds really build; breakdowns really thin out. Don't raise everything at once: contrast makes intensity legible.

• Under speech, keep the speech-critical register (roughly 300 Hz–3 kHz) clear: simpler lines, sustained

material, rests. The music opens up in the gaps between lines and after the last word.

• The chart is built from the scale's own chords; borrowed colours where they serve the story. Vary the harmony

across sections, and slow the harmonic rhythm to suspend time, speed it up to drive.

• Roles: 3 to 10, each on its own MIDI channel 1–16 (drums on 10), with a vivid name, a job in the story, and

an instrument_idea (the instruments stage chooses the real sound).

• Songs: the genre_signature names which of the genre's unforgettable elements carry the hook and the beat; the

length comes from the genre's length unless the brief fixed it (then "length_fixed": true).

node $M plan checks it and writes music/BLUEPRINT.md, the readable version (it's what the owner reviews for a song).

5. The score: music/score.strudel.js, notes only

setcpm(96 / 4)                                      // BPM / beats per bar: one cycle is one bar
const BARS = 48
const cue = (p) => p.filterWhen((t) => t < BARS)    // every part ends with the piece (Strudel loops otherwise)
const lift = "<0!12 1!20 0!4 1!12>"                  // a section mask, one value per bar: off 1-12, on 13-32, off 33-36, on 37-48
sub:   cue(note("<d2 c2 [g1 a1] d2>").velocity(.8)).midichan(1)
pad:   cue(note("<[d3,f3,a3,c4,e4] ...>").velocity(.6)).midichan(2)
hook:  cue(note("<[a4 ~ c5 d5] ...>").velocity("<.7 .85>").mask(lift)).midichan(3)
drums: cue(note("[36 ~ ~ 36 38 ~ 36 ~], [42*8]").velocity("<.8 .95>").mask(lift)).midichan(10)

• Every part is a labelled block, wrapped in cue(…), with .midichan(n) = its role's channel.
• Only music goes in the score: pitches, rhythm, .velocity() (the playing dynamic: it goes into the MIDI),

structure (masks, <…> sequences, arrange()). No .s(), filters, effects or .gain(): those belong to the instruments, and the engine warns when the score sets them.

• Drums on channel 10 are General MIDI numbers (36 kick, 38 snare, 42 closed hat, 46 open hat, 49 crash, 39 clap,

45 tom): the instruments stage gives each number a sound, and the MIDI pack maps them straight into any drum rack.

• Sections are masks or <…> sequences with one value per bar, aligned to the blueprint's bars. Name each mask

after its section.

6. Check it, and capture the MIDI

node $M check [--video-sec <animatic length> --fps <fps>]

It reports the harmony lock, every hit's distance from its frame, parts that play past the end, roles that never play, and the intensity curve: a sparkline per section, asked vs measured. Iterate until 0 problems. Then read the curve against the story: does it rise where it should, does it hold back under the busiest narration?

When it passes, the score's notes become the MIDI: music/notes.json (every channel's notes in beats, with velocity) and out/music/midi/ (one .mid per channel, the whole arrangement in one file, tempo, meter, section names and hit points as markers, and CUE-SHEET.md). From here on the MIDI is the music: the instruments are chosen from it, and the owner can take it into any DAW. The check also prints where each channel plays (lowest, middle, highest note).

7. Instruments: a sound for every channel

node $M instruments lists each channel's range from the MIDI and the built-in sounds. Write music/instruments.json: