Give it one still of a person and a short speech clip; get back a video of that person
speaking the clip, with the clip as its soundtrack. Pin the same still as first and
last frame, and every render opens and closes on the identical pose, so chunks of a
long track join without a visible cut. H3 re-renders the whole face, so skin comes out
smoother than a photographed take. When a take already exists and only the mouth is
wrong, use heygen-v3-resync-existing-take (best quality) or
veed-v2-resync-existing-take (cheapest) instead: they keep the take's skin, light and
performance.
Inputs
- The still. For a presenter that will be composited into layouts, make it a 3:4 portrait with plain backdrop either side wide enough for open-arm gestures (they reach about ±35% of the frame height from the centre line; 9:16 clips them). Fit it to 768×1024 for 768P or 1080×1440 for 1080P. A 3/4 view works.
- The audio. Mono WAV of one speaker, 5 to 15 s. Cut it with
ffmpeg-cut-audio-window. The hard floor is 2 s (shorter is rejected); pad a short line with silence to 5 s, never take a longer window, or the next speaker's words get lip-synced too. Drive it with an isolated voice, not a mix with music. - A long track: cut it into chunks of 5 to 14 s at pauses, a pause after
.?!beating any pause inside a sentence (python-plan-lipsync-chunks). At each cut the person returns to the pinned pose, which reads as a breath between sentences and as a hitch mid-sentence. - The words of the clip, from the script or the transcript (they go in the prompt).
Run it
Call describe("fal/minimax/h3-max/image-to-video") before the first run in a job.
run("fal/minimax/h3-max/image-to-video", {
"image_url": "<job>/stills/presenter_34_e.png",
"end_image_url": "<job>/stills/presenter_34_e.png",
"target_audio_url": "<job>/talking_head/chunk03.wav",
"prompt": "<H3 prompt, see below>",
"duration": 9.84,
"resolution": "768P",
"prompt_expansion_mode": "disabled",
"seed": 7
}, into="<job>/talking_head", unit="chunk03_h3lipsync", tag="chunk03")
duration: the clip's length in seconds, clamped to 2 to 15.resolution:480P,768Por1080P, uppercase P. Use768Pwith a 3:4 still.prompt_expansion_mode: "disabled": the schema requires the field, and fal's "balanced" rewrite adds a soundtrack.- Target audio works on image-to-video only, never on reference-to-video. For a scene
where several people speak, use
h3-max-dialogue. - Chunks are independent: start them all at once, one
runeach withwait: false, thentasks(wait: true, calls: [...], until: "any"). Check each chunk as it lands (Check it) and call again with the ids still running.
Prompt
Load h3-prompt-format first and write an image-to-video prompt: what the still shows,
and how the person moves while speaking. Then, for this mode:
- Put the words in the prompt. The prompt never hears the audio. Told only "she
speaks to camera", the writer produced "her lips remain completely closed throughout"
and H3 rendered a silent face with the audio playing over it. Append the clip's words:
She speaks these words aloud to camera, her mouth moving in clean sync with the audio on every word and closing only in the pauses: "Before you open that link, stop and check who really sent it."The prompt then carries a<d>dialogue block and the lips move. - Gestures: ask for natural presenter gestures (hands up on key points, one held while the other drops, back to the sides between points), and tie them to words with a per-chunk beat ("On 'stop' both hands open outward"). Never ask for finger counts: a "three fingers on 'Finally'" beat produced a hand the person rejected.
- Don't name a removed accessory, even negatively ("no pin or badge" grew a pin by 1 s). Describe the clean state instead: "a plain, unadorned knit, the same from the first frame to the last".
What you get
- A 24 fps MP4 with the clip as its audio. It runs 0.05 to 0.7 s past
duration, so the true pinned end pose lies past the cut; cut it to the clip's length. - Cost is per second by tier, whatever the canvas shape, and the canvas follows the
still: 768P is half of 1080P (prices in
h3-max-image-to-video). 1080P is 768P plus latent refinement.
| Render | Face height | Relative | Price |
|---|---|---|---|
| 16:9 1080P | 167 px | 1.00 | full |
| 3:4 768P | 157 px | 0.93 | half |
| 3:4 1080P | 219 px | 1.31 | full |
3:4 768P puts about the same pixels on the face as 16:9 1080P for half the price; 3:4 1080P is the upgrade if a layout shows the person large.
Tips and traps
- Lips run 3 to 5% slow on some chunks (3 of 17 on one track: 3.3%, 3.8%, 5.5%).
The take starts in sync and is 200 to 500 ms late by the end of a 10 s chunk, with
video and audio the same length, so a duration check can't see it and a quick watch
misses it. The slope was identical on every seed: don't re-roll. Check every take
with
hn4a-lipsync-check; a steady drift is retimed for free (python-assemble-lipsync-chunks,retime: −9 → +511 ms became −9 → +22 ms), and a lag that jumps about gets a mouth re-sync (veed-v2-resync-existing-take). - Plastic skin. H3 smooths facial hair, skin texture and hair strands, and leaves skin shiny. Compare against a non-lip-synced render on a close-up before picking; for a close-up that must keep its texture, re-sync a real take instead.
- Changing the still re-renders every chunk, since each is pinned to it. Remove a small thing from the still with a local patch, never a full-frame edit.
- A hand still dropping at the cut needs a re-render with a new
seed; a drifting take is also late back to the pose, so retime it first. - H3 reaches for a raised index finger on some lines however it's directed. Accept the least-bad take rather than looping.
- Joining chunks: on one frame grid for the whole track, with a 5-frame dissolve
at each seam and the track under it (
python-assemble-lipsync-chunks,join). Trimming each chunk with-tdrifted 313 ms over 17 chunks, and renders differ by about 1% zoom, so a hard cut pops.
Check it
- Opens on the still, back in the pose at the cut, the mouth moving, and in sync
over time against the chunk's WAV:
python-assemble-lipsync-chunkscheckruns all of them per chunk. H3 re-renders the face, so a pinned opening reads 0.94–0.98 against the still (greyscale SSIM); a talking mouth moves 7–11 and a silent one about 1, which means the words were missing from the prompt; good takes sit between −83 and +60 ms. - Put the takes to the person with
review_choose, the still asoriginal, and record the pick.


