Make someone speak on screen. Write the audio first (guide-voice-a-script, or the
isolated voice from the source), then pick the route by the outcome wanted. Ask first:
is there a take whose skin, light and performance must stay, or are you starting from
a still, and how many people are in the shot?
- A still and one speaker (a presenter, an avatar, a line): lip-sync the still.
- A good take whose mouth is off or whose words changed: fix only the mouth.
- A scene with several people, blocking and voice-over: ref2vid with the audio.
Prices are as of 2026-10; get exact ones from describe(model) before a batch.
Routes
| Route | Pick it when | Cost and time | Quality | Limits | Load |
|---|---|---|---|---|---|
| A still plus audio (H3 Max i2v lip sync) | One speaker from a still; a long script cut into chunks at sentence pauses, the same still pinned at both ends of every chunk | Low per second, by resolution tier; fast | Good motion; re-renders the whole face (plastic, shiny skin, smoothed beards). Lips run 3–5% slow on some chunks | Audio 5–15 s per render, padded with silence | h3-max-lipsync-a-still |
| Mouth-only fix, HeyGen v3 Precision | A take worth keeping; the best-looking mouth | ≈ $0.10 per second of video | Keeps the take's skin, light and performance; picked over VEED and H3 on a 3/4 close-up | The take's length | heygen-v3-resync-existing-take |
| Mouth-only fix, VEED v2 | Same, cheapest; H3 lag that a free retime (python-assemble-lipsync-chunks) can't remove | ≈ $0.07 per second | Keeps the take; returns 25 fps, convert back to 24 | The take's length | veed-v2-resync-existing-take |
| ref2vid with audio, Seedance 2.5 | A scene from a still, references and the track's window; locked framing; voice-over | High per render; minutes | Holds the frame; mouths closed on voice-over; adequate lips | Audio up to 30 s, 29 references; no end frame | seedance-2-5-dialogue |
| ref2vid with audio, H3 Max | Same, when the shot needs bigger blocking or better lips, or an end frame | Low; fast | Better lip sync, stronger blocking; pushes in, may cut to a new angle on long lines, can duplicate a person | Audio up to 15 s, 9 references; the only model with end frame plus audio | h3-max-dialogue |
For a scene, render each shot on both ref2vid models and let the person pick; picks split about evenly. For a locked single-person talking head, Seedance holds best.
Preparing the audio
- Transcribe it with
elevenlabs-speech-to-text, thenpython-read-word-timingsfor words, speaker turns and pauses. Diarization clusters voices, not people. - From a mix, isolate the voice first (
elevenlabs-voice-isolation). Cut where the next speaker starts by ear, from a ladder of trims a few hundredths of a second apart: STT and loudness minima were each ~0.1 s off. - Cut windows with
ffmpeg-cut-audio-windowat pauses: 0.2 s before the shot's first word, ending 0.2 s before the next shot's first word, or halfway into a shorter pause. A ref2vid window under 2 s is padded to 3 s. H3 lip sync wants 5–15 s: pad with silence, never take a longer window. - Chunks for a long track: 5–14 s, cut in a pause after a sentence where one fits,
every cut on the frame grid (
python-plan-lipsync-chunks). At each cut the speaker returns to the pinned pose, which reads as a breath between sentences and a hitch mid-sentence. The renders are checked and joined on that grid bypython-assemble-lipsync-chunks; a presenter for a whole video isguide-present-with-an-avatar. - Plan silence into the track wherever the picture needs action time (an exit, a transition) before rendering any video.
Directing ref2vid scenes
- Cast only the people physically in the frame; a face on a screen gets no reference.
- Pin each speaker by screen position at every line ("the woman on the RIGHT").
- Screens are a fixed state ("a paused still"), never a replay.
- Voice-over: say nobody on screen talks.
- Time-cue the action to the transcript ("1.7–9.2 s: the man counts the tests").
- One seed for the whole job, so a one-line change is a true A/B.
- Seedance with a recorded line: give a clip-length recording with each line at its
real time, and bind each line to its span in the prompt (
seedance-2-5-prompt-format). With a short untimed reference it re-performs the line at its own pace: up to 1.3 s late and ~10% long. Takes rendered that way can be cut to the line withpython-find-line-in-render. - The models re-voice the line, and the mouth follows their version. Assemble each clip with its own audio; never lay the track under a re-voiced clip (Seedance drifted up to 0.44 s mid-shot). The words came back verbatim on every pick.
For H3 Max lip sync
- Put the spoken words in the prompt, or the prompt says the lips stay closed and H3 renders a silent face. A talking mouth crop moves 5–8 (mean frame difference); a silent one ~1.
- Render the still as 3:4 portrait at 768P for presenters: about the same pixels on the face as 16:9 1080P at half the price. 9:16 clips wide gestures.
- Keep gestures minimal unless asked. When asked, direct a beat per line, and ban finger counting outright: asked for fingers on "First / Second / Finally", H3 held up three on a line with no third item.
Checking sync
Run hn4a-lipsync-check on every lip-synced take. It reads the mouth's lag every 0.2 s
and flags spans late by more than +90 ms or early by more than −150 ms, sustained for
1 s (~10 s per 10 s clip). Good H3 takes sit at −83 to +60 ms. H3's tempo drift (in
sync at the start, 200–500 ms late by the end of a 10 s chunk) hit 3 of 17 chunks and
nobody spotted it in a quick watch.
Gates, each a review with sound on: voices → stills → video → export. The
dialogue track has none: once the voices are picked, it goes on when its transcript
checks out. Give the source as original and last round's take as previous, the new
takes as options.
Revisions
Fix a chunk or a shot end to end: re-voice its audio, re-render it, re-assemble. Don't
splice audio or patch video by hand. H3's lip timing comes from the audio and is the
same on every seed, so a drifting chunk won't fix with re-rolls: retime a steady drift
(python-assemble-lipsync-chunks), and re-sync its mouth with VEED when the lag isn't a
steady slope.
Avoid
- Sync 3: it left the face unchanged (output ~identical to input).
- Re-rolling H3 drift: same slope on every seed.
- Attaching every character to every shot: wrong people in rooms, lines on wrong faces.
- Laying the track under re-voiced clips.
- Patching audio or video by hand on a revision.
Related
guide-voice-a-script (the audio), guide-choose-a-video-model, guide-animate-a-shot
(silent shots), guide-recast-a-person (new people, then new voices).


