OneShot MV turns a song and one line about its world into a lip-synced music video, locally in ComfyUI
OneShot MV is a free, open-source ComfyUI pack and Studio app that takes a song and one line describing its world, invents a cast, plans every shot around the song's energy and renders a lip-synced music video. It runs locally on open-weights models: Krea 2 for the portraits and LTX-2.3 for the singing clips.
What it does
It analyses the song (beats, bars, sections, loudness, the drop, vocal and drum stems), turns your line into a World with people, outfits, places and light, and cuts by energy: long shots in quiet parts, a cut every bar in verses, a new face on the beat in the chorus. Krea 2 paints a different invented performer for each line, a vision check redoes wrong portraits (extra people, writing, objects from the wrong period), LTX-2.3 makes each portrait sing the isolated vocal, and a take check re-shoots frozen or drifting takes and weak lip sync scored with SyncNet. It then cuts on the beat, adds drum-driven effects, a grade, grain and a vignette, and exports an MP4, an EDL and a Final Cut Pro XML for Premiere Pro or DaVinci Resolve. You can re-shoot any single shot with a note and re-cut the post processing in about two minutes.
Results and limits
The author built and tested it on an NVIDIA RTX PRO 6000 (96 GB) with the full model set: a 32-second video took about 70 minutes, a 1:51 video about 6 hours 45 minutes, and re-shooting one shot about 10 to 15 minutes. With ComfyUI held to 24 GB on the same card a 1:04 video took about 3 hours 35 minutes, and the README says a real 24 GB card will be slower still.
Worth knowing
The default model set is made for 24 GB GPUs (RTX 3090, 4090 or 5090) and wants at least 64 GB of system RAM; it is about 73 GB to download (the full bf16 set, for 48 GB+ cards, about 144 GB). It needs ComfyUI-LTXVideo, KJNodes and RES4LYF, and runs from a Studio page inside ComfyUI, as a workflow, or from the Comfy SDK. Output is 16:9, 9:16 or 1:1 at 1080p. The code is MIT; the README says Krea 2 allows commercial use only for companies under US$1M annual revenue, LTX-2.3 asks you to label output as AI-generated, and the optional insightface framing check is non-commercial. Every face is invented; use music you have the rights to.