Seedance 2.5: dialogue shot from a still and a voice track

Give it the shot's still, a reference picture or two per person on screen, and the shot's piece of the recorded dialogue; get a clip that opens on the still, holds the framing, and has the people in frame speak the lines with their mouths following the audio. It is the steadier of the two dialogue models: the camera stays put, blocking is literal and restrained, and off-screen voiceover leaves every mouth closed. Use h3-max-dialogue when the shot needs bigger performance, sharper lip sync on a large face, or an end frame together with audio, and its window fits in 15 s. For a scene, render both and pick per shot (guide-lipsync-video).

Inputs

The still, matched to its script shot by what it shows, never by file name.
The cast: only the people physically in frame. Someone only on a screen in the shot (a TV, a video call) gets no picture and no label; the still carries them. A reference invites the model to put that person in the room.
Reference pictures: single views from each character sheet (python-split-character-sheet), at most 1,920 px on the long side (bigger ones carry texture the model copies as noise). Up to 5 subjects and 29 pictures.
The audio: the shot's window of the track as mono WAV, cut at pauses, never through a word (ffmpeg-cut-audio-window). 1.8 to 30 s; pad a window under 2 s with silence to 3 s, never widen it (extra words get spoken too). Split a longer shot at a pause.
Each line's start and end in the window, from the track's word timings (python-read-word-timings), for the timed prompt.
One seed for the whole job, so a one-line change is a true A/B: identical inputs reproduced identical clips.

Run it

For the timed prompt, pad the window to the clip's whole-second length so the line times are clip times: exec ffmpeg -v error -y -i <job>/audio/shot10.wav -af apad=whole_dur=5 -t 5 <job>/audio/shot10_5s.wav

Call describe("fal/bytedance/seedance-2.5/us/reference-to-video") before the first run in a job, then:

run("fal/bytedance/seedance-2.5/us/reference-to-video", {
  "prompt": "<Seedance prompt, see below>",
  "image_urls": ["<job>/stills/shot10.jpg", "<job>/refs/dana_1.jpg",
                 "<job>/refs/dana_2.jpg", "<job>/refs/ravi_1.jpg"],
  "audio_urls": ["<job>/audio/shot10_5s.wav"],
  "duration": "5",
  "aspect_ratio": "16:9",
  "resolution": "720p",
  "generate_audio": true,
  "bitrate_mode": "high",
  "seed": 1234
}, into="<job>/runs/run03", unit="shot10_seedance", tag="shot10")

image_urls: the still first (it becomes @Image1), then each subject's pictures in the prompt's order.
aspect_ratio: the nearest of 21:9, 16:9, 4:3, 1:1, 3:4, 9:16 to the still. Reference-to-video doesn't infer it.
duration: the audio's length rounded up to whole seconds, "4" to "30".
resolution: 720p for runs; 1080p when the shot is final.
There is no end frame in this mode. For a transition that must land on the next shot's still, use h3-max-dialogue.

Prompt

Load seedance-2-5-prompt-format first and use the reference-to-video shape with the timed audio lines (From 0.09 to 1.48 seconds, @Image2 speaks {…} from @Audio1…). With an untimed window Seedance re-performs the line at its own pace: it started up to 1.3 s late and ran about 10% long, and no re-cut fixes the drift. Use the untimed line only for off-screen voiceover. Write the direction with these clauses, each answering a failure seen on a real scene:

Camera: "Static shot: the camera stays locked on the opening framing for the whole clip, with no zoom, push-in, pull-back, pan or cut." Every shot, unless the action moves the camera.
Positions for two-handers, repeated at every line: "@Dana is the woman on the RIGHT; @Ravi is the man on the LEFT. They never swap places." Speaker names alone swapped lines between people.
Screens as a fixed state: "The wall TV shows a paused still" or "is switched off, solid black", plus "nothing on it plays, moves, blinks or speaks". Never "a replay".
Action time-cued to the transcript: "1.7–9.2 s: @Ravi counts the files on the desk."
Voiceover: "Nobody on screen speaks: every mouth in frame stays closed."
"Only the person speaking moves their mouth; everyone else keeps their mouth closed while listening." and "No music, no subtitles, no on-screen text."
When the clip runs past the window: "play them at natural real-time speed, then hold the final moment with only small natural movement until the clip ends."
Don't stage a beat of action before the first line unless the original has one.

What you get

An MP4 that opens on the still, with Seedance's own performance of the lines: it re-voices the reference audio and the mouth follows its version, not the track. The words come back verbatim, but the timing drifts from the track (up to 0.44 s mid-shot; a 2.2 s window in a 4 s clip started 0.5 s late with an invented pause).

Tips and traps

Keep the clip's own audio. Never lay the original track under it. Cut the picks together each with its own sound, letting a shot run past its window when its re-voiced line does.
Layout drift. Seedance keeps the still's layout but can open slightly zoomed or shifted, and sometimes on a wider framing. Measure it (python-measure-layout-drift), then accept or counter-crop; for a wide opening, state the framing in the scene ("from mid-shin up; never pulls back") and re-render.
Content filter rejections on silent or ambient-only shots: run again as the next take.
Seedance follows time-cued action more literally than H3 Max.
Cartoon characters: on a drawn scene Seedance opened at 0.95–0.98 against the still, once 0.60, where H3 Max held 0.994–0.997 on the same shots; prefer h3-max-dialogue for cartoons, and check every opening.

Check it

First frames after every run: still, frame 0 and last frame side by side, with frame_alignment (python-check-pinned-frames). Under 0.95 is not opening on the still (a bad opening scored 0.68, good ones 0.98 and up); re-render with a framing sentence.
Lip sync over time: hn4a-lipsync-check.
Review unmuted, beside the H3 Max take of the same shot.

Hypernatural