Log in
WorkflowBy HypernaturalVersion 7, updated

Recast dialogue video

hn/workflow-recast-dialogue-video

Replace one actor in a finished dialogue video with a new character and a new voice: cast and perform the voice, recast that character's stills in their own outfit, re-animate only their shots with the dialogue as reference audio, and export each pick with its own audio for the editor. One variant per new actor; every other shot is reused as delivered.

SKILL.md

The person brings a finished dialogue video (the cut, a still per shot, the script), the new character (a character sheet, written guidance or both) and the number of variants. You deliver, per variant, the re-animated shots of that character, each with its own audio, the new dialogue track with that audio spliced in, a timing sheet, and one audio file per reused shot with its position on the track, for the editor who cuts it all into the existing footage.

Only the shots that show the replaced character are re-imaged and re-animated. Gate every stage on the person's review: voice, stills, video, export. Never move on unasked.

Steps

  1. The script. If there is no script.json, make one from the video (guide-make-a-script-from-source).
  2. Cast the voice (guide-cast-a-voice), once per new character. Gate: the person picks the voice.
  3. Perform the script per variant (guide-voice-a-script), on eleven_v4. Transcribe every take (elevenlabs-speech-to-text): the words, each character's pace, the pauses the notes mention. Hand-write the tags for per-line notes. Wherever the picture needs action time (an exit, a transition), add silence to the track now, before any video. No gate: the voice was picked in step 2, so a take that passes the check goes on.
  4. Map stills to shots by content, never by filename. Record the table with the job's notes: shot, source still, whether the character appears. For a shot with no still, propose a similar one to reuse. Gate: the person approves the table.
  5. Scrub the footage for the old actor before any stills work: every shot, including backs of heads, shoulders, reflections and screens. When a still and the footage disagree, recast from a frame of the footage (exec ffmpeg -ss <t> -i <cut> -frames:v 1).
  6. Recast the stills (guide-recast-a-person, nano-banana-recast-in-own-outfit): single views from the sheet (python-split-character-sheet), the back view added for a shot from behind, 2 takes per shot. Remove a badge or pin with nano-banana-remove-small-object, on the stills and the reference views both. Gate: a review_choose per shot, all in the same turn, its takes as the options and the delivered still as original.
  7. Animate the character's shots (guide-lipsync-video, ref2vid with audio). Cut each shot's window from the take's word times (python-read-word-timings): 0.2 s before its first word, to 0.2 s before the next shot's first word, or halfway into a shorter pause. A line that runs across a cut can move to the next shot. Render every shot on both seedance-2-5-dialogue and h3-max-dialogue, one seed for the job. Give H3 Max the transitions that must land on the next shot's still: it is the only model that takes an end frame with audio. Start every render of a run at once, check each clip as it lands (tasks with until: "any") for opening on its still (python-measure-layout-drift; under 0.95 is a miss), and re-render a miss with its framing stated while the rest still render. Gate: a review_choose per shot, all in the same turn, its clips from every run so far as the options, with their sound. Iterate only the shots with notes, into the next run.
  8. Export. Each pick keeps the audio the model rendered: both models re-voice the line and the mouth follows their version, not the track. Write each pick with its own audio, a dialogue.wav with that audio spliced in at each shot's window, a timing sheet (shot, file, start on the track), and one audio file per reused shot with its position on dialogue.wav. Transcribe the exported track to check every placement. Then deliver the round's deliverables (core skill).

Choices this workflow makes

  • Both video models on every shot: picks split about evenly. H3 Max won the transition and most to-camera close-ups; Seedance holds a locked frame and keeps mouths shut on voiceover.
  • If the person wants to see blocking before paying for both, run H3 Max alone first; a draft approves composition and timing, not motion.
  • Recast in the character's own outfit, not the old actor's wardrobe.
  • The ref2vid casting, speaker pinning and gesture rules are guide-lipsync-video's, Directing ref2vid scenes; on this job's to-camera shots, hands stayed at the sides.
  • Name a removed accessory nowhere in the video prompt: describe what is there ("a plain, unadorned knit, the same from the first frame to the last").

Dead ends

  • Laying the new track under the clips, and attaching every character to every shot: the failures guide-lipsync-video now guards against (out of sync; lines on the wrong faces).
  • Trusting the delivered stills to show every appearance: a back-of-head shot had no still with the old actor in it, and was caught only after export.
  • "No pin or badge" in the scene: H3 Max grew a small dark pin by 1 s.
  • Counting on fingers on a closing to-camera line: read as fussy and was cut.

Worked example

The Globex training drama, Episode 1 recast: two variants, Presenter Dana and Presenter Sam, replacing the original presenter. The script had 52 lines; the eleven_v4 takes came back word-perfect at about 2.9 words a second for both presenters.

  • Stills: a woman-to-man swap at a wide conference table worked first time, 2 takes per shot. Shot 16, the presenter's back of head, was missed at first; recast from the cut's frame with the back view, it worked first time.
  • Video: 8 presenter shots per variant, 4 runs, both models per shot. H3 Max took the shot 18 transition and most to-camera close-ups; Seedance took shots 11 and 12 in both variants. The words came back verbatim on all 16 re-voiced picks.
  • Shot 18: the presenter stands, turns to camera and walks forward while the room fades to black, landing on shot 19's still (H3 Max end frame plus audio). The walk needed 2.4 s of silence added to the track after the line "That's the whole trick."
  • Shot 23, Seedance, run 1: opened on a wider framing than the still. "From mid-shin up; never pulls back" fixed it.

FAQs

Replace one actor in a finished dialogue video with a new character and a new voice: cast and perform the voice, recast that character's stills in their own outfit, re-animate only their shots with the dialogue as reference audio, and export each pick with its own audio for the editor. One variant per new actor; every other shot is reused as delivered.

Install Hypernatural Desktop, a free Mac app, and connect it to Claude Code, Codex or Claude Desktop. Then describe the video you want and your agent finds the workflow itself, or paste this page's link (https://hypernatural.ai/skill/hn/workflow-recast-dialogue-video) and it installs hn/workflow-recast-dialogue-video and follows it.

Nothing. The workflow is free and runs with tools on your own Mac, like ffmpeg.

It calls 15 smaller skills for its steps, including Make a script from source, Cast a voice and Voice a script. Each one is listed on this page, and your agent loads them as the job needs them.

Yes. Ask your agent to adjust it for your job, or to save its own version. You can publish your version under your own handle so anyone can install it.

Go make something cool.

Install the app, open your agent, describe the video. That's the whole onboarding.

Sticker HypernaturalSticker BannerSticker TVBoy