LSDub re-syncs a singer's lips to a new vocal by warping the face's own mouth, shot by shot across a whole music video
LSDub is a free, open-source ComfyUI node pack and toolkit that fixes lip sync in music videos without re-rendering the shot or painting a new mouth. It lifts the singer's own teeth, lips and tongue from their best frames and warps them to the vocal, then scores every shot so you know which ones worked.
What it does
It finds the singer in each shot and plans the dub from the lyrics, then gates every sung word by face size: close-ups (240 px or more) get the full treatment, medium faces (120 to 240 px) are cropped, upscaled, warped and scaled back, and wide shots are flagged for a recut rather than faked. Mic and hand masks keep the warp behind the mic, and each painted frame is checked against the last four to catch ghost lips and jumps. A QA suite measures sync, lag in frames, tooth size and motion, and a scorecard ranks the film shot by shot, worst first.
Results and limits
In the authors' Pleiadians Don't Dance demo (64 cuts, 8 of them dubbable performance shots), LSDub 0.1.4 measured sync of 0.81 to 0.93, and fixing continuity took one cut from 0.14 back up to 0.97.
Worth knowing
It is built for music videos; spoken dialogue and film dubbing are still in development. It uses LTX-2.5 and open-weights tools only, with no partner APIs; the scan, paint and QA steps run on any machine, and the heavier checks use the GPU when one is free. The node pack, four example workflows and a demo pack are Apache-2.0; LTX-2.5 needs a paid licence from US$10M annual revenue. An entry in Comfy's Open Source Higgsfield challenge.