Log in
TechniqueBy HypernaturalVersion 2, updated

VEED v2: re-sync the mouth of an existing take, cheapest

hn/veed-v2-resync-existing-take

Re-sync only the mouth of an existing video take to its audio with VEED Lipsync v2 (video-to-video lip sync), the cheapest mouth-only fix at $0.07/s. The fix for an H3 Max lip-sync chunk whose lips are out of sync in a way a free retime (python-assemble-lipsync-chunks) can't remove, flagged by hn4a-lipsync-check. Returns 25 fps, so convert back to the take's rate. For a close-up where the mouth is watched, use heygen-v3-resync-existing-take.

SKILL.md

Give it a take and the audio it must speak; get the take back with only the mouth redone, the take's skin, light and performance kept. Its main job is fixing H3 Max lips that are out of sync: on some chunks they run 3 to 5% slow, identically on every seed, so a re-roll can't fix it. A steady slope is retimed first, for free (python-assemble-lipsync-chunks, retime); this is for a lag that jumps about, or a retimed take that still flags.

When to use this and when its sibling:

VEED v2 (this)HeyGen v3 Precision (heygen-v3-resync-existing-take)
Price$0.07/s$0.10/s
Wins ondrift on a talking-head chunk; any fix where cost matters more than the last bit of mouth detaila close-up or 3/4 view where the mouth is watched: HeyGen was picked over VEED there
Returns25 fps whatever went inthe take's length

Inputs

  • The take (MP4) and its frame rate (ffprobe-inspect-media).
  • The audio it must speak, mono WAV starting at the take's first frame. For an H3 chunk it's the chunk's own WAV, the one the take was rendered on. Otherwise cut it with ffmpeg-cut-audio-window.

Run it

  1. Trim the take to the audio's length, with the audio muxed in. frames = round(audio seconds × fps):

    exec ffprobe -v error -select_streams v:0 -show_entries stream=r_frame_rate -of csv=p=0 chunk08_h3lipsync_a.mp4
    exec ffprobe -v error -show_entries format=duration -of csv=p=0 chunk08.wav
    exec ffmpeg -v error -y -i chunk08_h3lipsync_a.mp4 -i chunk08.wav -map 0:v -map 1:a \
      -vf "trim=end_frame=217,setpts=PTS-STARTPTS" \
      -c:v libx264 -crf 14 -preset slow -pix_fmt yuv420p -c:a aac -b:a 256k chunk08_trim.mp4
    
  2. Call describe("fal/veed/lipsync/v2") before the first run in a job, then:

    run("fal/veed/lipsync/v2", {
      "video_url": "<job>/talking_head/chunk08_trim.mp4",
      "audio_url": "<job>/talking_head/chunk08.wav"
    }, into="<job>/talking_head/raw", unit="chunk08_veed", tag="chunk08")
    
  3. Convert it back from 25 fps to the take's rate on the take's frame grid, cut to frames, with the audio muxed in again:

    exec ffmpeg -v error -y -i raw/chunk08_veed_a.mp4 -i chunk08.wav -map 0:v -map 1:a \
      -vf "fps=24:round=near" -frames:v 217 \
      -c:v libx264 -crf 14 -preset slow -pix_fmt yuv420p -c:a aac -b:a 256k chunk08_h3lipsync_b.mp4
    

    fps=24:round=near picks the nearest 25 fps frame for each 24 fps slot; a 217-frame take came back 217 frames, identical to a fix made by hand. Don't blend or interpolate frames.

Save the result as the chunk's next variant letter and record what it came from (the take, veed_v2, the audio), so a later round knows it's a re-sync.

What you get

The take with a re-synced mouth, at the take's frame rate and frame count. Cost is $0.07 per second of video.

Tips and traps

  • If the chunk's audio changed (a re-voiced chunk), re-render the chunk on the new audio rather than re-syncing a take made on the old one.
  • The accepted re-sync of a drifting chunk measured −4 to −127 ms over its length: inside the tolerance, with the mouth a little early, as good H3 takes are.

Check it

  • Run hn4a-lipsync-check on the result against the same audio: it must come back with no flagged spans. Check every re-sync, not only the first.
  • Count frames: equal to frames from step 1.
  • Watch it once at playback size next to the flagged take.

FAQs

Re-sync only the mouth of an existing video take to its audio with VEED Lipsync v2 (video-to-video lip sync), the cheapest mouth-only fix at $0.07/s. The fix for an H3 Max lip-sync chunk whose lips are out of sync in a way a free retime (python-assemble-lipsync-chunks) can't remove, flagged by hn4a-lipsync-check. Returns 25 fps, so convert back to the take's rate. For a close-up where the mouth is watched, use heygen-v3-resync-existing-take.

Install Hypernatural Desktop, a free Mac app, and connect it to Claude Code, Codex or Claude Desktop. Then describe the video you want and your agent finds the skill itself, or paste this page's link (https://hypernatural.ai/skill/hn/veed-v2-resync-existing-take) and it installs hn/veed-v2-resync-existing-take and follows it.

It runs fal/veed/lipsync/v2. Hypernatural Desktop has every one of them built in, so there are no API keys to set up, though you can bring your own.

The skill is free. You only pay for the AI generations it runs, and your agent shows you the price of every batch before it spends anything. Editing, captions and titles run on your Mac at no cost.

Yes. Ask your agent to adjust it for your job, or to save its own version. You can publish your version under your own handle so anyone can install it.

Go make something cool.

Install the app, open your agent, describe the video. That's the whole onboarding.

Sticker HypernaturalSticker BannerSticker TVBoy