calesthio/generative-media-skills/audioshake-stem-separation

AudioShake stem separation

AudioShake is a hosted audio source-separation service: it takes one mixed recording and returns isolated component tracks ("stems"). It does not generate, synthesize, mix, or master audio — it decomposes an existing mix. This skill covers driving its developer API, choosing the right separation targets for a production job, judging and repairing the resulting stems, the rights questions that separation raises, and when to reach for a local open model instead.

All API facts below were verified against developer.audioshake.ai on 2026-07-10. Endpoints, model names, credit rates, and limits are volatile — re-check the live models / billing / rate-limits reference pages and the OpenAPI spec (https://developer.audioshake.ai/api-reference/openapi.json) before relying on a specific number in production.

When this skill applies

Use it when the job is to pull apart a recording:

• Music stems — isolate vocals, drums, bass, guitar, piano, keys, strings, or winds for remixing, karaoke, sampling, sync, immersive/Atmos re-authoring, or practice/minus-one tracks.
• Dialogue / Music / Effects (DME) — split a film, TV, or podcast mix into clean dialogue, a music+effects bed, or an effects-only bed for localization, foreign-language dubbing, M&E (music-and-effects) deliverables, or archival restoration where the original stems are lost.
• Multi-speaker — separate an interview, panel, meeting, or overlapping-dialogue recording into one stem per speaker.
• Lyric transcription / alignment — line-level lyric transcripts or word-level karaoke timing.
• Content analysis / cleanup — detect or identify music inside long-form content (rights clearance), or denoise/dereverb speech.

Do not use it to write, generate, or synthesize audio, to compose music, to mix or master, or to add effects. Those are generative or engineering tasks, not separation. If the user already has clean stems and wants them combined or processed, that is out of scope.

API model: the Tasks API

Base URL: https://api.audioshake.ai (verified 2026-07-10). Auth: every request carries the header x-api-key: <key>. Keys are created in the dashboard (dashboard.audioshake.ai). New accounts receive 10 free credits on signup (first-party claim, verified 2026-07-10).

Note: AudioShake replaced its older Jobs API (/job, /upload, per-request callbackUrl) with the Tasks API. The Tasks API is the current interface: POST /assets instead of /upload, POST /tasks instead of /job, GET /tasks/{id} instead of /job/{id}, and webhooks registered once via POST /webhooks instead of a per-job callback. If you find code using /job, it is on the legacy path.

Job lifecycle

1. Provide the source audio. Either pass a public `url` (HTTPS) directly, or upload a local file with POST /assets and pass the returned `assetId`. url and assetId are mutually exclusive per task.
2. Create a task with POST /tasks, listing one or more targets. Each target is { "model": "<name>", "formats": ["<fmt>", ...] }. A single task takes 1–20 unique targets and runs them together — this is both cheaper in round-trips and the recommended way to avoid rate limits.
3. Wait for completion by either polling GET /tasks/{id} or registering a webhook. Each target moves through processing → completed | error.
4. Download outputs from the output array. Download URLs are presigned and expire ~1 hour after the task completes — fetch and store them promptly, or write directly to your own bucket (see writeDestination).

Request body reference (verified 2026-07-10)

Response fields

id, createdAt, completedAt (or null), cost (credits actually charged), targets[] (each with status, output files, and error), a top-level output[] of downloadable files, and error (code + message, null on success).

Limits and formats (verified 2026-07-10)

• Rate limit: ~60 requests/second. On 429, back off exponentially, and prefer batching many targets into one task and using webhooks over tight polling.
• Input audio: WAV, AIFF, FLAC, MP3, AAC (up to 192 kHz; 16/24-bit WAV recommended).
• Input video: MP4, MOV (AAC or PCM audio; the video track is ignored — audio is extracted).
• Output: WAV, MP3, FLAC, AIFF; MP4 (separated audio in the original container); JSON/SRT/TXT for text.
• Not supported: DRM/encrypted content, and multi-channel/surround formats (e.g. 5.1) — downmix to stereo first if needed.
• Duration caps by model: multi_voice ≤ 1.5 hours; transcription and alignment ≤ 45 minutes. Other separation models have no documented cap but cost scales with length.

The model catalog

Credits are charged per minute of source audio, per target model, rounded up to the next whole minute (verified 2026-07-10). So a 3:10 song billed for two targets is charged as 4 min × 2 targets. Rates below are the documented per-minute costs.

Music / instrument stems — 1.0 credit/min each

vocals (all sung voice, lead + backing) · vocals_lead (lead only) · vocals_backing (harmonies, ad-libs, choir) · instrumental (full mix minus vocals) · drums · bass · guitar (all guitar) · guitar_electric · guitar_acoustic · piano (acoustic piano) · keys (keyboard family, incl. electric/digital pianos) · strings · wind (woodwind + brass) · other (everything except vocals/drums/bass) · other-x-guitar (everything except vocals/drums/bass/guitar).

Post-production DME — 1.5 credits/min each

dialogue (clean speech stem) · effects (ambience + SFX bed, dialogue and music removed) · music_fx (music + effects, dialogue removed).

Speech models

multi_voice (10.0 credits/min, ≤ 1.5 h) — one stem per detected speaker, keeping each speaker isolated even through overlap. speech_denoise (1.5/min) — remove background noise, hum, interference. speech_dereverb (2.0/min) — remove room reverb/echo for a drier, closer sound.

Copyright-compliance / analysis

music_detection (0.5/min) — return time ranges where music is present. music_identification — identify music and return track metadata. music_removal — keep speech + effects, remove background music.

Lyric transcription — 1.0 credit/min each (≤ 45 min)

transcription — line-level timestamped lyric transcript (JSON; convertible to SRT/VTT). alignment — precise word-level and line-level timestamps; requires a supplied transcript (transcriptUrl/transcriptAssetId).

Choosing targets for a production job

The single most common mistake is requesting the wrong granularity — either paying for stems nobody will use, or requesting a coarse stem when the job needs a fine one. Match targets to the deliverable:

• Karaoke / instrumental backing: vocals + instrumental covers it. If you also need on-screen timed lyrics, add transcription (or alignment if you already have accurate lyrics and only need timing). Requesting drums/bass/etc. as well wastes credits unless the product actually plays them separately.
• Remix / production stems: request the specific instruments the remixer will touch (vocals_lead, drums, bass, guitar, keys…). Use other / other-x-guitar to sweep up the remaining energy so the sum reconstructs the mix — otherwise instruments you didn't request vanish.
• "Remove one instrument" (minus-one / practice track): request that instrument with residual: true to get the complement in one target, rather than requesting all the other stems and re-summing.
• Localization / dubbing (M&E): request dialogue (translation reference) + music_fx (the bed to layer the new voiceover onto). Add effects separately only if the workflow needs music and effects independently (e.g. remixing the music but keeping SFX).
• Interview / podcast cleanup: if speakers overlap and you need them apart, multi_voice. If speech is buried in noise or a boomy room, chain speech_denoise and/or speech_dereverb first, then separate.
• Rights clearance on long-form content: music_detection to find where music appears, then music_identification on those ranges; music_removal if you must deliver a music-free version.

Prefer the finest stem the deliverable justifies, and always include a residual/other target when the stems must re-sum to the original.

Reviewing separated stems

Separation is never perfect. The following are production heuristics for judging output, not documented guarantees.

Listen to each stem soloed, and also summed back together, checking for:

• Bleed / crosstalk — traces of another source leaking into a stem (a snare ghost in the "vocals" track, vocal breath in "drums"). Most audible on soloed stems and in quiet passages.
• Musical-noise / "watery" artifacts — the burbling, underwater texture from spectral holes, worst on vocals and busy other stems, and on heavily-compressed source (MP3 at low bitrate).
• Transient smearing / pre-echo — softened or ghosted drum hits and pluck attacks; check drums, bass, and plucked guitar.
• Phase / nulling problems — sum the stems and null-test against the original: a good multi-target set reconstructs the mix closely. Large residual energy means a source went missing (you didn't request an other/residual target) or a stem is phase-rotated.
• Missing energy — sustained pads, reverb tails, and room tone often collapse into other; if you didn't request it they are simply gone.

Repair strategies

1. Feed a cleaner source. Separation quality tracks input quality directly — a 24-bit WAV master separates far better than a 128 kbps MP3. Never up-transcode a lossy file and expect it to help.
2. Cover the residual. If summed stems don't reconstruct the mix, add an other/other-x-guitar target or use residual: true so no energy is silently dropped.
3. Split then clean, in the right order. For speech, run speech_denoise/speech_dereverb before multi_voice when the recording is degraded; for music stems, separate first, then repair the isolated stem with a dedicated tool (e.g. iZotope RX spectral repair) rather than the source.
4. Use finer targets to reduce bleed. vocals_lead + vocals_backing often leaves less bleed than a single vocals when only the lead is needed.
5. Re-derive DME by subtraction. If dialogue is clean but effects has bleed, deriving effects as residual of dialogue+music can beat the direct effects model on some material — A/B them.
6. Escalate to manual finishing. Persistent artifacts are a mix/restoration problem: hand the stem to a spectral editor rather than re-running the API repeatedly.

Rights, consent, and privacy