version: "1.0.0" name: audiocpp-tts-controls description: "Advanced synthesis on the loaded GPU host TTS model via the raw audiocpp gateway (/tts): seed, language, voice-design instructions, builtin/voice-pack speakers, and multi-speaker dialogue with overlapping interruptions. None of which the built-in GuideAnts audio tools expose." metadata: guideants: enabled: true display_order: 32 requires_toolsets: [sandbox]

audio.cpp synthesis controls (experimental)

Paths — fixed layout, do not probe or re-derive. The sandbox CWD is the notebook's output directory. This skill's scripts live under Skills/audiocpp-tts-controls/scripts/ relative to it. Write deliverables with bare filenames (e.g. -o scene.wav); never prefix with Output/.

Product TTS is {text, voice, speed}. This skill calls raw /tts/v1/audio/speech on the GPU host. Deliverables are WAVs written to the CWD (bare filenames).

Environment (required for PC → the GPU host)

AUDIOCPP_SKILL_BASE_URL=http://<gpu-host-lan-ip>:8112/audiocpp-skill
AUDIOCPP_SKILL_TOKEN=<same as the GPU host GA_AUDIOCPP_SKILL_TOKEN>

A TTS model must already be loaded on the GPU host via GuideAnts Settings. Chatterbox (ResembleAI/chatterbox) is the known-good catalog entry for cloning / voice-pack presets.

Long-form single-speaker narration (video scripts)

Skill users: write the final narration script as plain text (markdown headers ok) and call narration.py. Do not chunk, concat, trim silence, or run preflight yourself — the tool owns all of that.

start runs preflight, creates a job, and spawns a detached background worker that synthesizes the full script. Each sandbox call returns immediately. Poll status until "status": "done". Use cancel if the user stops the turn or asks to abort.

python3 Skills/audiocpp-tts-controls/scripts/narration.py start script.txt \
  -o narration.wav \
  --voice narrator
 
python3 Skills/audiocpp-tts-controls/scripts/narration.py status narration.wav

Repeat status until "status": "done". Job state lives in .audiocpp-narration/ beside the output. Status responses report job-level progress only (progress, elapsed_seconds, eta_seconds) — never chunk indices or segment counts.

To replace a finished or failed job:

python3 Skills/audiocpp-tts-controls/scripts/narration.py start script.txt \
  -o narration.wav --voice narrator --force

To abort a running job:

python3 Skills/audiocpp-tts-controls/scripts/narration.py cancel narration.wav

Short utterances

For a single line or short clip, use engine_tool.py speech directly:

python3 Skills/audiocpp-tts-controls/scripts/engine_tool.py speech "Hello there" \
  -o out.wav \
  [--seed 42] \
  [--language de] \
  [--instructions "a calm, deep narrator voice"] \
  [--voice Vivian]

• --instructions only on vdes-task models (VoiceDesign catalog entry).
• --language values are family-specific; engine errors name what is valid.
• Same text + seed + model ⇒ same audio.

Voices

python3 Skills/audiocpp-tts-controls/scripts/engine_tool.py voices

Voice ids come from the voice-pack baked into the GPU host AI image at /opt/guideants/voice-pack/ (or a custom mount at /opt/guideants/custom-voice-pack). The engine resolves a --voice id to a reference clip server-side — you do NOT stage the clip for pack voices. A custom user voice (e.g. a cloned reference added to the pack's manifest.json) works the same way: pass its voiceId as --voice.

Choosing a voice for a script (important)

The voice-pack manifest.json records each voice's language and gender. Match the voice's `language` to the script's language. Chatterbox is an English model: an in-distribution pair (English reference + English text) locks onto one stable voice. An out-of-distribution pair (e.g. a French- reference ff_* voice speaking English) produces an unstable, text- sensitive accent — the same speaker drifts into different accents across lines, and a seed cannot fix this (a seed only controls sampling noise on top of a stable distribution). If a voice sounds like a different person on each line, check its language in the manifest first, then swap for a matching-language voice.

Multi-speaker dialogue (overlap / interruptions)

python3 Skills/audiocpp-tts-controls/scripts/multi_speaker.py \
  scene.json -o scene.wav \
  [--model chatterbox] \
  [--seed-map '{"narrator":1001,"alice":3003}']

scene.json is an array of lines:

[
  {"voice": "narrator",  "text": "What's the air speed velocity of an unladen swallow?"},
  {"voice": "bm_george", "text": "African or European?", "overlap_ms": 400},
  {"voice": "bf_alice",  "text": "Oh no, you two are doing the swallow thing again.", "overlap_ms": 600}
]

• voice / text — required.
• overlap_ms — how many ms this line **starts before the previous line

ends. `0` (default) = clean turn-taking; `>0` = the new speaker talks over / interrupts. For a realistic interruption the overlap should be ~1s+** so real words from both speakers are audible at once; 200–300 ms only overlaps a trailing consonant and does NOT read as an interruption.

• seed — per-line override. Omit to use an auto per-speaker seed.

How the mix works (do not change these)

1. Two tracks on one timeline. Each line is placed at its computed start

offset on a shared buffer and additively mixed where they overlap. No crossfade, no truncation — both voices are genuinely audible together.

1. Preserve the native sample rate. The TTS engine emits 24 kHz mono.

Writing the output at a different rate (e.g. 48 kHz) plays the audio at the wrong speed and mangles it. Always pass the source framerate through.

1. One seed per speaker. Chatterbox is non-deterministic per call. Assign

each speaker a fixed seed (the script auto-derives one from the voice name by default) so a speaker sounds consistent across all their lines. This only works for in-distribution (matching-language) voices — see above.

Realistic interruption beats

To make interruptions sound natural, write the interrupted speaker's line as a full flowing sentence that is still going when the other person cuts in (so the overlap zone contains real words from both), rather than a truncated fragment. A short overlap_ms on a clean turn is fine for pacing, but the "talk-over" moments should have overlap_ms ≥ ~1000.

When this isn't enough

• Loaded model rejects a voice preset id → confirm the pack is mounted and

the id is in manifest.json; for a brand-new custom voice add it to the pack and rebuild/remount, or stage a clip and use audiocpp-voice-clone (--voice-ref).

• No TTS loaded on the GPU host → blocked; say so.

Reporting

End by telling the user what worked and what was blocked, quoting preflight evidence from narration.py start when using long-form narration. For multi-speaker output, state the total duration, per-speaker seeds, and which lines carry overlap_ms. For long narration, quote status (progress, elapsed_seconds, eta_seconds, final result.duration_seconds) — not chunk or segment details.