Log in
TechniqueBy HypernaturalVersion 7, updated

H3 Max: lip-sync a still

hn/h3-max-lipsync-a-still

Lip-sync a still to an audio clip with H3 Max image-to-video and target audio: one locked presenter or speaker still plus 5 to 15 s of speech becomes a talking head that opens and closes on the still. Use for avatar and presenter videos, a long voice track cut into chunks, or a line on a shot with no usable take; to fix only the mouth of an existing take, use heygen-v3-resync-existing-take or veed-v2-resync-existing-take instead.

SKILL.md

Give it one still of a person and a short speech clip; get back a video of that person speaking the clip, with the clip as its soundtrack. Pin the same still as first and last frame, and every render opens and closes on the identical pose, so chunks of a long track join without a visible cut. H3 re-renders the whole face, so skin comes out smoother than a photographed take. When a take already exists and only the mouth is wrong, use heygen-v3-resync-existing-take (best quality) or veed-v2-resync-existing-take (cheapest) instead: they keep the take's skin, light and performance.

Inputs

  • The still. For a presenter that will be composited into layouts, make it a 3:4 portrait with plain backdrop either side wide enough for open-arm gestures (they reach about ±35% of the frame height from the centre line; 9:16 clips them). Fit it to 768×1024 for 768P or 1080×1440 for 1080P. A 3/4 view works.
  • The audio. Mono WAV of one speaker, 5 to 15 s. Cut it with ffmpeg-cut-audio-window. The hard floor is 2 s (shorter is rejected); pad a short line with silence to 5 s, never take a longer window, or the next speaker's words get lip-synced too. Drive it with an isolated voice, not a mix with music.
  • A long track: cut it into chunks of 5 to 14 s at pauses, a pause after . ? ! beating any pause inside a sentence (python-plan-lipsync-chunks). At each cut the person returns to the pinned pose, which reads as a breath between sentences and as a hitch mid-sentence.
  • The words of the clip, from the script or the transcript (they go in the prompt).

Run it

Call describe("fal/minimax/h3-max/image-to-video") before the first run in a job.

run("fal/minimax/h3-max/image-to-video", {
  "image_url": "<job>/stills/presenter_34_e.png",
  "end_image_url": "<job>/stills/presenter_34_e.png",
  "target_audio_url": "<job>/talking_head/chunk03.wav",
  "prompt": "<H3 prompt, see below>",
  "duration": 9.84,
  "resolution": "768P",
  "prompt_expansion_mode": "disabled",
  "seed": 7
}, into="<job>/talking_head", unit="chunk03_h3lipsync", tag="chunk03")
  • duration: the clip's length in seconds, clamped to 2 to 15.
  • resolution: 480P, 768P or 1080P, uppercase P. Use 768P with a 3:4 still.
  • prompt_expansion_mode: "disabled": the schema requires the field, and fal's "balanced" rewrite adds a soundtrack.
  • Target audio works on image-to-video only, never on reference-to-video. For a scene where several people speak, use h3-max-dialogue.
  • Chunks are independent: start them all at once, one run each with wait: false, then tasks(wait: true, calls: [...], until: "any"). Check each chunk as it lands (Check it) and call again with the ids still running.

Prompt

Load h3-prompt-format first and write an image-to-video prompt: what the still shows, and how the person moves while speaking. Then, for this mode:

  • Put the words in the prompt. The prompt never hears the audio. Told only "she speaks to camera", the writer produced "her lips remain completely closed throughout" and H3 rendered a silent face with the audio playing over it. Append the clip's words: She speaks these words aloud to camera, her mouth moving in clean sync with the audio on every word and closing only in the pauses: "Before you open that link, stop and check who really sent it." The prompt then carries a <d> dialogue block and the lips move.
  • Gestures: ask for natural presenter gestures (hands up on key points, one held while the other drops, back to the sides between points), and tie them to words with a per-chunk beat ("On 'stop' both hands open outward"). Never ask for finger counts: a "three fingers on 'Finally'" beat produced a hand the person rejected.
  • Don't name a removed accessory, even negatively ("no pin or badge" grew a pin by 1 s). Describe the clean state instead: "a plain, unadorned knit, the same from the first frame to the last".

What you get

  • A 24 fps MP4 with the clip as its audio. It runs 0.05 to 0.7 s past duration, so the true pinned end pose lies past the cut; cut it to the clip's length.
  • Cost is per second by tier, whatever the canvas shape, and the canvas follows the still: 768P is half of 1080P (prices in h3-max-image-to-video). 1080P is 768P plus latent refinement.
RenderFace heightRelativePrice
16:9 1080P167 px1.00full
3:4 768P157 px0.93half
3:4 1080P219 px1.31full

3:4 768P puts about the same pixels on the face as 16:9 1080P for half the price; 3:4 1080P is the upgrade if a layout shows the person large.

Tips and traps

  • Lips run 3 to 5% slow on some chunks (3 of 17 on one track: 3.3%, 3.8%, 5.5%). The take starts in sync and is 200 to 500 ms late by the end of a 10 s chunk, with video and audio the same length, so a duration check can't see it and a quick watch misses it. The slope was identical on every seed: don't re-roll. Check every take with hn4a-lipsync-check; a steady drift is retimed for free (python-assemble-lipsync-chunks, retime: −9 → +511 ms became −9 → +22 ms), and a lag that jumps about gets a mouth re-sync (veed-v2-resync-existing-take).
  • Plastic skin. H3 smooths facial hair, skin texture and hair strands, and leaves skin shiny. Compare against a non-lip-synced render on a close-up before picking; for a close-up that must keep its texture, re-sync a real take instead.
  • Changing the still re-renders every chunk, since each is pinned to it. Remove a small thing from the still with a local patch, never a full-frame edit.
  • A hand still dropping at the cut needs a re-render with a new seed; a drifting take is also late back to the pose, so retime it first.
  • H3 reaches for a raised index finger on some lines however it's directed. Accept the least-bad take rather than looping.
  • Joining chunks: on one frame grid for the whole track, with a 5-frame dissolve at each seam and the track under it (python-assemble-lipsync-chunks, join). Trimming each chunk with -t drifted 313 ms over 17 chunks, and renders differ by about 1% zoom, so a hard cut pops.

Check it

  1. Opens on the still, back in the pose at the cut, the mouth moving, and in sync over time against the chunk's WAV: python-assemble-lipsync-chunks check runs all of them per chunk. H3 re-renders the face, so a pinned opening reads 0.94–0.98 against the still (greyscale SSIM); a talking mouth moves 7–11 and a silent one about 1, which means the words were missing from the prompt; good takes sit between −83 and +60 ms.
  2. Put the takes to the person with review_choose, the still as original, and record the pick.

Workflows that use this skill

FAQs

Lip-sync a still to an audio clip with H3 Max image-to-video and target audio: one locked presenter or speaker still plus 5 to 15 s of speech becomes a talking head that opens and closes on the still. Use for avatar and presenter videos, a long voice track cut into chunks, or a line on a shot with no usable take; to fix only the mouth of an existing take, use heygen-v3-resync-existing-take or veed-v2-resync-existing-take instead.

Install Hypernatural Desktop, a free Mac app, and connect it to Claude Code, Codex or Claude Desktop. Then describe the video you want and your agent finds the skill itself, or paste this page's link (https://hypernatural.ai/skill/hn/h3-max-lipsync-a-still) and it installs hn/h3-max-lipsync-a-still and follows it.

It runs MiniMax H3 Max. Hypernatural Desktop has every one of them built in, so there are no API keys to set up, though you can bring your own.

The skill is free. You only pay for the AI generations it runs, and your agent shows you the price of every batch before it spends anything. Editing, captions and titles run on your Mac at no cost.

Yes. Ask your agent to adjust it for your job, or to save its own version. You can publish your version under your own handle so anyone can install it.

Go make something cool.

Install the app, open your agent, describe the video. That's the whole onboarding.

Sticker HypernaturalSticker BannerSticker TVBoy