H3 Max: lip-sync a still

Give it one still of a person and a short speech clip; get back a video of that person speaking the clip, with the clip as its soundtrack. Pin the same still as first and last frame, and every render opens and closes on the identical pose, so chunks of a long track join without a visible cut. H3 re-renders the whole face, so skin comes out smoother than a photographed take. When a take already exists and only the mouth is wrong, use heygen-v3-resync-existing-take (best quality) or veed-v2-resync-existing-take (cheapest) instead: they keep the take's skin, light and performance.

Inputs

• The still. For a presenter that will be composited into layouts, make it a 3:4 portrait with plain backdrop either side wide enough for open-arm gestures (they reach about ±35% of the frame height from the centre line; 9:16 clips them). Fit it to 768×1024 for 768P or 1080×1440 for 1080P. A 3/4 view works.
• The audio. Mono WAV of one speaker, 5 to 15 s. Cut it with ffmpeg-cut-audio-window. The hard floor is 2 s (shorter is rejected); pad a short line with silence to 5 s, never take a longer window, or the next speaker's words get lip-synced too. Drive it with an isolated voice, not a mix with music.
• A long track: cut it into chunks of 5 to 14 s at pauses, a pause after . ? ! beating any pause inside a sentence (python-read-word-timings, chunks). At each cut the person returns to the pinned pose, which reads as a breath between sentences and as a hitch mid-sentence.
• The words of the clip, from the script or the transcript (they go in the prompt).

Run it

Call describe("fal/minimax/h3-max/image-to-video") before the first run in a job.

run("fal/minimax/h3-max/image-to-video", {
  "image_url": "<job>/stills/presenter_34_e.png",
  "end_image_url": "<job>/stills/presenter_34_e.png",
  "target_audio_url": "<job>/talking_head/chunk03.wav",
  "prompt": "<H3 prompt, see below>",
  "duration": 9.84,
  "resolution": "768P",
  "prompt_expansion_mode": "disabled",
  "seed": 7
}, into="<job>/talking_head", unit="chunk03_h3lipsync", tag="chunk03")

• duration: the clip's length in seconds, clamped to 2 to 15.
• resolution: 480P, 768P or 1080P, uppercase P. Use 768P with a 3:4 still.
• prompt_expansion_mode: "disabled": the schema requires the field, and fal's "balanced" rewrite adds a soundtrack.
• Target audio works on image-to-video only, never on reference-to-video. For a scene where several people speak, use h3-max-dialogue.
• Chunks are independent: start them all at once, one run each with wait: false, then tasks(wait: true, calls: [...], until: "any"). Check each chunk as it lands (Check it) and call again with the ids still running.

Prompt

Load h3-prompt-format first and write an image-to-video prompt: what the still shows, and how the person moves while speaking. Then, for this mode:

• Put the words in the prompt. The prompt never hears the audio. Told only "she speaks to camera", the writer produced "her lips remain completely closed throughout" and H3 rendered a silent face with the audio playing over it. Append the clip's words: She speaks these words aloud to camera, her mouth moving in clean sync with the audio on every word and closing only in the pauses: "Before you open that link, stop and check who really sent it." The prompt then carries a <d> dialogue block and the lips move.
• Gestures: ask for natural presenter gestures (hands up on key points, one held while the other drops, back to the sides between points), and tie them to words with a per-chunk beat ("On 'stop' both hands open outward"). Never ask for finger counts: a "three fingers on 'Finally'" beat produced a hand the person rejected.
• Don't name a removed accessory, even negatively ("no pin or badge" grew a pin by 1 s). Describe the clean state instead: "a plain, unadorned knit, the same from the first frame to the last".

What you get

• A 24 fps MP4 with the clip as its audio. It runs 0.05 to 0.7 s past duration, so the true pinned end pose lies past the cut; cut it to the clip's length.
• Cost is per second by tier, whatever the canvas shape, and the canvas follows the still: 768P is half of 1080P (measured at $0.052/s against $0.104/s on a promo rate, $0.08 against $0.16 at list). 1080P is 768P plus latent refinement.

3:4 768P puts about the same pixels on the face as 16:9 1080P for half the price; 3:4 1080P is the upgrade if a layout shows the person large.

Tips and traps

• Lips run 3 to 5% slow on some chunks (3 of 17 on one track: 3.3%, 3.8%, 5.5%). The take starts in sync and is 200 to 500 ms late by the end of a 10 s chunk, with video and audio the same length, so a duration check can't see it and a quick watch misses it. The slope was identical on every seed: don't re-roll. Check every take with hn4a-lipsync-check and re-sync a flagged one with veed-v2-resync-existing-take.
• Plastic skin. H3 smooths facial hair, skin texture and hair strands, and leaves skin shiny. Compare against a non-lip-synced render on a close-up before picking; for a close-up that must keep its texture, re-sync a real take instead.
• Changing the still re-renders every chunk, since each is pinned to it. Remove a small thing from the still with a local patch, never a full-frame edit.
• A hand still dropping at the cut (frame match under 0.95 at the chunk's end) needs a re-render with a new seed.
• H3 reaches for a raised index finger on some lines however it's directed. Accept the least-bad take rather than looping.
• Joining chunks: cut every chunk on one frame grid for the whole track: chunk n covers frames round(start·fps) to round(end·fps). Trimming each chunk with -t rounds each up by part of a frame and drifted 313 ms behind the audio over 17 chunks; on the grid the worst error is half a frame. A ~1% zoom difference still pops at a hard cut, so dissolve 5 frames over the outgoing clip's tail; the incoming clip starts exactly at its chunk start, so sync holds. Lay the full track under the joined video.

Check it

1. First frame against the still, and the frame at the clip's length against the still: both 0.99 or above (python-check-pinned-frames). Framing held at 0.984 or above with at most 1.5% zoom on a proven track.
2. The person is talking: a silent face with the audio playing over it means the words were missing from the prompt.
3. Lip sync over time: hn4a-lipsync-check on the take against the chunk's WAV. Good H3 takes sit between −83 and +60 ms.
4. Put the takes on a review page with the still, and record the pick.