ElevenLabs Instant Soundtracks scores a video and places sound effects in one pass
Instant Soundtracks in ElevenCreative takes an uploaded video, writes an original score for it and places sound effects at the moments that call for them. The effects land on their own layers, and the API can return music and effects as separate files.
What it does
You upload a video and the model scores it with original music, then watches the footage for cues and generates sound effects at those timestamps, for example a whoosh timed to a logo at the end of an ad. It can use the clip's existing audio as a reference, reading its genre, tempo, crescendos or instrumentation to shape the new score, and it can shift the music's mood from scene to scene.
In the ElevenCreative Music Video Editor the effects sit on separate layers, so you can adjust their timing, add more, swap the music or refine single moments without starting over.
Results and limits
Sonetho ran the API on a 12.7-second video: music alone took about 10 seconds and 193 credits, and music plus separate sound effects took about 19 seconds and 347 credits. ElevenLabs says timing around cuts and transitions is tuned for short-form video.
Worth knowing
It's available in the ElevenCreative Music Video Editor, Creative Engine for ads, the ElevenMusic platform and the ElevenCreative Studio API, which has three new parameters: add_sfx, separate_tracks and use_audio_as_reference. With separate_tracks=true the API returns a ZIP with the music and the effects as separate files. A request takes up to 10 videos, 200 MB and 10 minutes of video combined.
Try it yourself
Upload a 30-second cut with at least two scene changes, generate a score with sound effects and separate tracks, then mute the music and judge the effects on their own, and the other way round.