gatewai-dev/framefields/framefields-compose

framefields-compose

Setup

Import everything from the `framefields` npm package. If the project doesn't have framefields set up yet, follow the framefields skill first (install, project scaffold, first render).

Overview

framefields is a code-first, deterministic video composition SDK executing natively on WebGPU (Dawn) in Node.js. It replaces browser-based rendering (Puppeteer/Chromium) with high-performance GPU pipelines, Yoga flexbox layout, Slug vector typography, 3D camera transforms, and reactive audio/signal drivers.

Every composition is defined as a tree of layout nodes compiled into a deterministic execution program: (composition, frame) → pixels.

Architecture & Core Concepts

1. Composition Lifecycle

A Composition instance defines canvas dimensions, frame rate, background, registered fonts, global audio tracks, and the layer hierarchy:

import { Composition, Layer, FontManager } from "framefields";
// Always register fonts before assembling layers
await FontManager.register({
  family: "Inter",
  source: "assets/fonts/Inter.ttf"
});
const comp = new Composition({
  width: 1920,
  height: 1080,
  fps: 30,
  durationMs: 10000, // or computeDuration()
  backgroundColor: "#0E0D0C",
  fonts: ["assets/fonts/Inter.ttf"],
});
comp.add(Layer.box({ /* ... */ }));
comp.add(Layer.audio("assets/score.mp3", { volume: 1 }));

2. Scene Architecture (Production Pattern)

For commercials, films, and multi-shot videos, break the project into discrete, modular scene functions using scene-local frames:

• Tokens first (`theme.ts`): Centralize color palette, typography, easing curves, and a tempo-locked beat grid (BEAT = Math.round(fps * 60 / BPM)). Never use arbitrary margins or ad-hoc hex codes.
• Scene modules (`scenes/hero.ts`): Each scene returns a full-frame root box with startFrame and durationFrames. Children position themselves relative to scene start (frame: 0 = scene onset).
• Master assembly (`film.ts`): Imports scenes, mounts them to comp.add(scene()), attaches the master soundtrack and composition-level grading/grain.

For the exported Scene / Sequence builder, scene windows are half-open: [startFrame, startFrame + durationFrames). A root layer starting at local frame 30 in a 60-frame scene defaults to 30 frames of duration. Explicit root-layer windows must fit their scene; invalid starts or overflow throw with the scene and layer IDs. Children and their keyframes remain local to their parent. Outgoing overlaps place the next scene and cannot exceed either adjacent scene's duration. The final scene's outgoing overlap has no destination and does not shorten the timeline. calculateTimeline().totalFrames is the maximum exclusive scene end.

// theme.ts
export const W = 1920;
export const H = 1080;
export const FPS = 30;
export const BEAT = 18; // 100 BPM at 30 fps
export const beats = (n: number) => Math.round(n * BEAT);
export const INK = "#0E0D0C";
export const PAPER = "#EEE9E0";
export const EMBER = "#FF5A1F";
export function scene(options: { id: string; from: number; to: number; children: unknown[] }) {
  return Layer.box({
    id: options.id,
    position: "absolute",
    x: 0,
    y: 0,
    width: W,
    height: H,
    startFrame: options.from,
    durationFrames: options.to - options.from,
    children: options.children as never,
  });
}

Layer Primitives (Layer.*)

All Layer.* builders return AnimatableNode<T> instances with chainable .animate(), .apply(), and effect methods.

Media time remapping

Layer.ambientLight({ color, intensity, id }) and Light.ambient({ color, intensity, id }) accept options objects like the other light helpers; the positional (color, intensity, options) form remains supported. Animation-builder instances passed through a layer's animation option are normalized to program specs, including CameraAnimation.

Video and audio layers accept timeRemap with an explicit durationFrames. Map frames are layer-local; sourceSec is absolute source time. Increasing source times play forward; decreasing source times play in reverse; equal source times hold the picture and silence the audio. Rate changes also change audio pitch. Do not combine this with trimStartSec.

const clip = Layer.video("assets/clip.mp4", {
  durationFrames: 90,
  timeRemap: { kind: "keyframes", keyframes: [
    { frame: 0, sourceSec: 1, interpolation: "smooth" },
    { frame: 30, sourceSec: 2 }, { frame: 45, sourceSec: 2 },
    { frame: 90, sourceSec: 1 },
  ] },
});
comp.add(clip);
const vision = comp.vision(clip);
// Or: timeRemap: { kind: "rate", sourceStartSec: 3, rate: -0.5 }

A signal can drive the clock instead: { kind: "signal", mode: "speed", signal, sourceStartSec } integrates playback rate (source seconds per second; negative reverses, 0 holds), and mode: "source-time" reads absolute source seconds. signal is a Signal or the name of a composition signal (comp.addSignal), read on the composition clock at every layer frame boundary and interpolated between them, so picture, audio and comp.vision share one clock.

const speed = Signal.state(1);
comp.onRequestFrame((ctx) => speed.set(ctx.frame < 48 ? 1 : 0.25));
comp.add(Layer.video("assets/clip.mp4", {
  durationFrames: 96,
  timeRemap: { kind: "signal", mode: "speed", signal: speed },
}));