Voice and audio

22 entries tagged voice and audio, newest first.

Filter: categories and tools

Categories

Tools

Show all 45 tools
All tools with counts >

  • Use case: a music video made fully locally with MiniMax H3, YuE2 and HiDream O1

    A creator made a three-minute music video using only open-weight models on their own machine: YuE2 for the song, HiDream O1 for character and world references, Qwen 3.8 27B to format prompts and MiniMax H3 for the video. The edit was done in DaVinci Resolve with only basic tools and no cloud AI.

    MiniMax H32 sources · x.com

  • Open ComfyUI workflow turns text into a lip-synced talking video with Drama 3 and MiniMax H3

    A free ComfyUI workflow from Skysay chains Fish Audio Drama 3 for expressive speech with MiniMax H3 for a lip-synced talking video, all on your own GPU. It ships as MIT-licensed graphs plus a script that runs both stages.

    drama3-h3-talking3 sources · github.com

  • Pika's ChatGPT plugin puts its video, image and sound apps inside the chat

    Pika launched a plugin for ChatGPT that runs its 40+ creation apps and 55+ image, video, music and speech models inside the chat. You can ask ChatGPT to bring up an editing timeline, and the plugin works with your existing Pika library.

    Pika4 sources · pika.art

  • Plenio 1.0 turns ComfyUI into a free local song studio

    Plenio is a free, open-source ComfyUI node pack that goes from a song idea to a mastered track on your own machine. A local language model writes the lyrics, YuE2 or MiniMax Music 3 performs them, and version 1.0 adds a browser player and lyric videos.

    Plenio Music Production System3 sources · github.com

  • WanGP v18.17 adds FLUX 3 Klein, Fish Audio Drama 3 and MOSS Image

    WanGP v18.17 brings FLUX 3 Image Klein, Fish Audio Drama 3 dialogue with voice cloning and MOSS Image 5B to the low-VRAM local app. It also adds a lighter H3 video decoder, INT8 math kernels and a Deepy Prime agent that plans a one-minute video from a brief.

    WanGP2 sources · x.com

  • ElevenLabs Instant Soundtracks scores a video and places sound effects in one pass

    Instant Soundtracks in ElevenCreative takes an uploaded video, writes an original score for it and places sound effects at the moments that call for them. The effects land on their own layers, and the API can return music and effects as separate files.

    ElevenLabs Instant Soundtracks3 sources · elevenlabs.io

  • Sarvam launches Bulbul V4 Flash for Indian languages

    Bulbul V4 Flash is a fast text-to-speech model with 83 voices across 11 Indian languages. It reaches first audio in under 250 ms and costs ₹20 per 10,000 characters.

    Bulbul V4 Flash1 source · sarvam.ai

  • Gnani Timbre v3.0 adds audio tags

    Timbre v3.0 is an Indian-language text-to-speech model that uses inline tags to make a voice laugh, sigh, whisper or pause. It launches with 13 voices in 10 languages.

    Gnani Timbre v3.01 source · gnani.ai

  • Mirelo SFX 2.0 rebuilds the sound of a video and keeps the dialogue

    Mirelo SFX 2.0 is a video-to-sound model that removes unwanted music from a clip, keeps the dialogue, and generates effects, foley and ambience synced to the action. It handles edits of up to 10 minutes in one pass and is now also available on Runway.

    Mirelo SFX 2.03 sources · mirelo.ai

  • Concat is a free, open-source CapCut alternative that runs its AI locally

    Concat is an open-source video editor for macOS, Windows, Linux and Android with auto-captions, text-to-speech with voice cloning and background removal that run on your own machine. It's in beta, has no watermark or account, and scripts or AI agents can drive it through a JSON-RPC API.

    Concat3 sources · github.com

  • Fish Audio releases Drama 3 emotive TTS

    Drama 3 is an expressive text-to-speech model in 83 languages that you direct with plain-language cues. It is available in the Fish Audio app and API, and as open weights under a community license.

    Fish Audio Drama 33 sources · fish.audio

  • HeyGen Video model arrives on Civitai

    HeyGen's first general-purpose video model makes 5 to 15 second clips with dialogue, ambience and sound effects generated in the same pass. It is now available in the Civitai generator for text, image or reference to video.

    HeyGen Video1 source · civitai.com

  • Kandinsky 6.0 Video released as open audio-video model

    Kandinsky 6.0 Video generates five-second clips with matching sound, including lip-synced speech, from text or an image. The Lite and Pro models, code and weights are released under the MIT license.

    Kandinsky 6.0 Video1 source · arxiv.org

  • Whistle: on-device speech to text in 16.9 MB

    Whistle is an open speech recognition model from Cactus that fits in a single 16.9 MB file and runs on a CPU without dependencies. It transcribes seven languages with word timestamps, which is handy for captions, and audio never leaves the device.

    Whistle2 sources · cactuscompute.com

  • Microsoft ships MAI-Voice-2.1 and Flash

    MAI-Voice-2.1 is a text-to-speech model covering 23 languages where a single voice keeps its identity across languages. The Flash variant targets high-volume use with about 150 ms end-to-end latency.

    MAI-Voice-2.11 source · microsoft.ai

  • ElevenLabs launches Eleven v4

    Eleven v4 is ElevenLabs' most expressive text-to-speech model, built for audiobooks, voiceovers, dubbing and character dialogue. A Turbo variant offers about 100 ms latency.

    Eleven v41 source · elevenlabs.io

  • Google releases Gemini 3.8 Flash TTS with voice design

    Gemini 3.8 Flash TTS and Flash-Lite TTS let creators design new voices from a text prompt and direct dialogue line by line in over 100 languages. They are available in Google AI Studio, the Gemini API and Google Vids, and every clip is watermarked with SynthID.

    Gemini 3.8 Flash TTS2 sources · blog.google

  • Nvidia expands AI for Media with lip sync and 8K upscaling

    Nvidia's AI for Media toolkit for broadcasters adds frame generation for slow motion, 8K upscaling and SDR-to-HDR conversion. It also adds real-time lip sync to translated audio and speaker detection.

    Nvidia AI for Media1 source · digitaltoday.co.kr

  • Nari Labs serves fast, low-cost Qwen3-TTS

    Nari Labs offers optimized versions of the open Qwen3-TTS and Qwen3-ASR speech models through a simple API. The company says they lead Coval's voice AI benchmark on the balance of quality, latency and cost.

    Nari Qwen3-TTS2 sources · narilabs.com

  • Premiere gets a Generative Media Tool in the timeline

    Adobe Premiere can now generate video and sound effects directly in the timeline, using Firefly or partner models such as Veo, Kling, Runway and Luma. After Effects also gets an AI Assistant in public beta.

    Adobe Premiere Generative Media Tool1 source · blog.adobe.com

  • LipDub 2.0 speeds up short-form localization

    LipDub 2.0 is a new lip sync model with no training step, making it 8x faster than LipDub 1.0. It handles single-speaker videos up to 2 minutes at 1080p in 80+ languages.

    LipDub 2.01 source · lipdub.ai

  • MiniMax H3 open weights run locally in ComfyUI

    MiniMax H3 is MiniMax's first open-weights video model, making clips with stereo sound up to 2K and 15 seconds. ComfyUI supported it on day one with pruned, quantized files that can run on a 12 GB RTX 3060.

    MiniMax H39 sources · blog.comfy.org

Questions and answers

Which text-to-speech models can I run locally?

Among the entries here, Fish Audio Drama 3 has weights on Hugging Face under the Fish Audio Community License. For speech-to-text, Whistle is a 16.9 MB model that runs on a CPU.

What is the cheapest TTS API for video voiceover?

Prices in our entries include Nari Qwen3-TTS from $5 per 1M characters, Fish Audio Drama 3 at $10 per 1M until November 12, and MAI-Voice-2.1 Flash at $15 per 1M. Quality and language support differ, so test with your own script.

How can I make captions for my video with AI offline?

Whistle transcribes seven languages with word timestamps on your own device, so audio never leaves it. Longer audio is processed in windows.

Can I clone a voice with Gemini TTS in Europe?

No, according to our Gemini 3.8 Flash TTS entry, voice replication isn't available in the EEA, the UK, Switzerland, India, Illinois or Texas. Voice design and the other features are available in Google AI Studio and the Gemini API.

How can I add music and sound effects to a finished video with AI?

ElevenLabs Instant Soundtracks scores an uploaded video and places timed sound effects on separate layers you can edit. Mirelo SFX 2.0 keeps the dialogue and rebuilds effects, foley and ambience for edits up to 10 minutes. To write and render songs locally, Plenio runs inside ComfyUI.