Cut clips to a voiceover

Give it the picked clips in shot order, the voice (one whole take, or one file per line) with its word times, and optionally a music bed; get the base cut: picture cut on the voice, the clips' own sound low under it, the bed ducked, at the aspect you ask for. Titles and captions go over this file later, in the compositor. For a cut on a song's bars instead of a voice, use python-cut-to-bar-grid; to put picked clips back into an existing timeline over its own audio, ffmpeg-splice-into-source-cut.

Inputs

• The clips, one per shot, at the cut's frame rate (H3 Max returns 24 fps). Render each a little longer than its window; the script trims, and holds the last frame when one is short. A clip at another rate is refused: conform it first (ffmpeg-frame-exact-trim).
• The voice and its word times. For one whole take, the take and its alignment or whisper JSON (python-read-word-timings reads either; guide-voice-a-script, A narrator for a short, makes the take). For one file per line (when lines must sit on shots of fixed length), the files and the time each starts.
• A bed, optional: 48 kHz WAV (elevenlabs-generate-music, What you get).
• cut/shots.json:

{
  "fps": 24, "size": [1920, 1080], "out": "cut/base_16x9.mp4",
  "words": "audio/voice_c.alignment.json",
  "voice": {"src": "audio/voice_c.wav", "at": 0.0},
  "bed": {"src": "audio/music/bed_a.wav", "volume": 0.3, "duck": true},
  "clip_volume": 0.2, "tail": 0.6,
  "shots": [
    {"clip": "video/shot01.mp4", "clip_9x16": "video/v/shot01.mp4", "phrase": "You've bookmarked"},
    {"clip": "video/shot02.mp4", "clip_9x16": "video/v/shot02.mp4", "phrase": "Globex Kitchen"},
    {"clip": "video/shot03.mp4", "clip_9x16": "video/v/shot03.mp4", "phrase": "Pick five", "from": 12}
  ]
}

• phrase: the first words of the shot's line, as the transcript or alignment spells them (a transcript writes "40,000" for "forty thousand": python-read-word-timings, Tips and traps). Each is found after the one before. start (seconds) instead of phrase pins a shot.
• from: the clip's first frame, to skip an idle opening (default 0).
• One file per line: replace words and voice with "lines": [{"src": "audio/line1.wav", "at": 0.2}, …], give every shot a start, and the cut's end.
• tail: seconds after the last word (0.6 for an ad; 2 s holds an end card); end overrides it.

Run it

Write references/cut_to_voice.md's script into the job's scripts/ folder as scripts/cut_to_voice.py, beside scripts/word_timings.py, and run it from the job folder:

exec python scripts/cut_to_voice.py cut/shots.json --plan
exec python scripts/cut_to_voice.py cut/shots.json
exec python scripts/cut_to_voice.py cut/shots.json --clip-key clip_9x16 --size 1080x1920 --out cut/base_9x16.mp4

--plan prints each shot's start and frame count, and which clips hold their last frame, without cutting: show it with the stills when the windows matter. Then loudness on the whole cut (ffmpeg-normalize-loudness: −16 LUFS, or −14 for shorts platforms, guide-pace-a-short-for-social). A re-rendered shot drops in by re-running the script.

What you get

An H.264 MP4 at the spec's size and rate, exactly round(end × fps) frames, stereo AAC, and cut/base_16x9_graph.txt (the filter graph it ran). The rules it applies:

• A shot starts at the midpoint of the gap before its line's first word (the first shot at 0), so the cut lands in the breath, not on the word.
• Every boundary is rounded once to the frame grid (ffmpeg-frame-exact-trim, The frame grid), so 10 shots never drift.
• Each clip is cut from frame from for exactly its frames, scaled to cover and centre-cropped with lanczos; a short clip holds its last frame (tpad clone).
• The clips' sound at clip_volume (0.2 under a voice; 0.35 for a cartoon whose effects carry the story) with 0.06 s fades at every cut; the voice at full level, mixed with amix normalize=0 so nothing is quietly halved.
• The bed under it all, ducked about 7 dB while the voice speaks (keyed on the voice with sidechaincompress), back up after the last word, faded out over the last second.

Tips and traps

• Never -t per clip, never one clip per file then concat. A 24 fps concat of separately cut clips drifted shot starts a few frames from the plan; the one-pass graph on frame counts doesn't.
• A long hook sentence pair can be two shots: give the second its own phrase.
• A clip that runs out holds its last frame, and --plan says by how many frames. Look at that shot in the cut: a held end pose can pass, a hold mid-action reads as a freeze, so re-render it longer rather than retiming it (ffmpeg-frame-exact-trim, Don't retime to fit).
• The clip's own sound stays low. H3 Max adds ambience and effects; at 0.2 they sit under the voice. Silent clips get silence, so the mix never shifts.
• One mono voice makes a mono cut in most tools: the script writes every input as stereo.
• A vertical cut is its own run from the native 9:16 clips (--clip-key), never a crop of the 16:9 cut (guide-pace-a-short-for-social).

Check it

• Frame count: exec ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of csv=p=0 cut/base_16x9.mp4 equals the total it printed.
• The frames either side of each cut on one sheet (python-contact-sheet): each shot starts on its own first frame.
• Every word: a transcript of the mix (whisper-cli-transcribe-with-word-times: a 16 kHz WAV, never the MP4) checked against the script (python-read-word-timings, check); numbers written as digits are the transcript's.
• Listen to the start and to one pause: the bed comes up between lines and never covers a word.