Log in
TechniqueBy HypernaturalVersion 2, updated

HeyGen v3 Precision: re-sync the mouth of an existing take

hn/heygen-v3-resync-existing-take

Re-sync only the mouth of an existing video take to new or corrected audio with HeyGen v3 Lipsync Precision (video-to-video lip sync), keeping the take's skin, light and performance. The best-looking mouth-only fix: picked over VEED v2 and H3 Max lip sync on a 3/4 close-up. Use when a well-acted take speaks the wrong line, drifts out of sync, or was re-voiced; for the cheapest fix use veed-v2-resync-existing-take.

SKILL.md

Give it a take and the audio it must speak; get the same take back with only the mouth redone. Everything else (skin texture, light, the performance, the framing) is the take's own, which H3 Max lip sync can't offer because it re-renders the whole face.

Pick between the two mouth-only siblings by what the shot needs:

HeyGen v3 Precision (this)VEED v2 (veed-v2-resync-existing-take)
Price$0.10/s$0.07/s
Wins ona close-up or 3/4 view where the mouth is watched: picked over VEED v2 and H3 lip sync on a 3/4 close-up of an angry linefixing drift on a talking head where the face is smaller; cheapest
Returnsthe take's length (with dynamic duration off)25 fps, always converted back

Sync 3 is not a sibling: on a 3/4-view face it returned the input nearly unchanged (PSNR 50 dB). Use h3-max-lipsync-a-still only when there is no usable take.

Inputs

  • The take (MP4). Note its frame rate with ffprobe-inspect-media.
  • The audio it must speak: mono WAV, the same length as the part of the take you keep, starting at the take's first frame. Cut it with ffmpeg-cut-audio-window: an isolated voice, silent from where the next speaker starts (picked by ear), so their words aren't lip-synced onto this mouth.

Run it

  1. Trim the take to the audio's length on its own frame grid, with the audio muxed in. frames = round(audio seconds × fps); read both with ffprobe:

    exec ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of csv=p=0 shot03_seedance_c.mp4
    exec ffprobe -v error -show_entries format=duration -of csv=p=0 shot03_line.wav
    exec ffmpeg -v error -y -i shot03_seedance_c.mp4 -i shot03_line.wav -map 0:v -map 1:a \
      -vf "trim=end_frame=33,setpts=PTS-STARTPTS" \
      -c:v libx264 -crf 14 -preset slow -pix_fmt yuv420p -c:a aac -b:a 256k shot03_seedance_c_trim.mp4
    
  2. Call describe("fal/fal-ai/heygen/v3/lipsync/precision") before the first run in a job, then:

    run("fal/fal-ai/heygen/v3/lipsync/precision", {
      "video_url": "<job>/video/shot03_seedance_c_trim.mp4",
      "audio_url": "<job>/audio/shot03_line.wav",
      "enable_dynamic_duration": false
    }, into="<job>/video/raw", unit="shot03_heygen", tag="shot03")
    

    enable_dynamic_duration: false matters: the default retimes the clip to the audio, and the shot's length is fixed.

  3. Put it back on the take's frame grid with the audio muxed in again, so it drops into the cut frame for frame (24 fps here; use the rate ffprobe gave):

    exec ffmpeg -v error -y -i raw/shot03_heygen_a.mp4 -i shot03_line.wav -map 0:v -map 1:a \
      -vf "fps=24:round=near" -frames:v 33 \
      -c:v libx264 -crf 14 -preset slow -pix_fmt yuv420p -c:a aac -b:a 256k shot03_heygen_a.mp4
    

Keep the take you started from: the re-synced file is a new variant letter, never an overwrite.

What you get

The take with a new mouth, the audio's length, at the take's frame rate after step 3. Cost is $0.10 per second of video.

Tips and traps

  • A re-sync fixes the mouth, not the delivery. If the performance is wrong (too calm, wrong beat), re-render the take first, then re-sync the best-acted one.
  • Re-voiced audio that runs longer than the take can't fit: re-render the shot. Never speed a clip up or slow it down to fit; it reads badly.
  • On a revision, re-sync from the take you started from, not from an earlier re-sync.

Check it

  • hn4a-lipsync-check on the result against the audio: no flagged spans.
  • Count frames (exec ffprobe -count_frames): it must equal frames from step 1.
  • Put the take and the re-sync to the person with review_choose, the original shot as original, and record the pick.

FAQs

Re-sync only the mouth of an existing video take to new or corrected audio with HeyGen v3 Lipsync Precision (video-to-video lip sync), keeping the take's skin, light and performance. The best-looking mouth-only fix: picked over VEED v2 and H3 Max lip sync on a 3/4 close-up. Use when a well-acted take speaks the wrong line, drifts out of sync, or was re-voiced; for the cheapest fix use veed-v2-resync-existing-take.

Install Hypernatural Desktop, a free Mac app, and connect it to Claude Code, Codex or Claude Desktop. Then describe the video you want and your agent finds the skill itself, or paste this page's link (https://hypernatural.ai/skill/hn/heygen-v3-resync-existing-take) and it installs hn/heygen-v3-resync-existing-take and follows it.

It runs fal/fal-ai/heygen/v3/lipsync/precision. Hypernatural Desktop has every one of them built in, so there are no API keys to set up, though you can bring your own.

The skill is free. You only pay for the AI generations it runs, and your agent shows you the price of every batch before it spends anything. Editing, captions and titles run on your Mac at no cost.

Yes. Ask your agent to adjust it for your job, or to save its own version. You can publish your version under your own handle so anyone can install it.

Go make something cool.

Install the app, open your agent, describe the video. That's the whole onboarding.

Sticker HypernaturalSticker BannerSticker TVBoy