Plan lip-sync chunks

Give it the voice track a presenter will speak and its word times; get the track cut into chunks for h3-max-lipsync-a-still (which takes 5 to 15 s of audio a render), each chunk's WAV, and chunks.json with each chunk's span, frames and words. Every cut sits in a real pause, and on the frame grid the assembly uses, so the renders join back into the track exactly (python-assemble-lipsync-chunks). For pauses alone, use python-find-speech-pauses; for a transcript's own gaps, python-read-word-timings.

Inputs

• The track: the exact WAV the presenter will speak, finished and leveled (any rate; the chunks keep it). Re-plan after any re-voice that changes its length.
• Its word times, in any shape python-read-word-timings reads. The TTS take's alignment is best (exact, spelled as the script); a speech-to-text JSON next; a whisper transcript only from the large model, which keeps the full stops (whisper-cli-transcribe-with-word-times).

Run it

Write references/lipsync_chunks.md's script into the job's scripts/ as lipsync_chunks.py, with speech_map.py (python-find-speech-pauses) and word_timings.py (python-read-word-timings) beside it, all as written:

exec python scripts/lipsync_chunks.py --track audio/track.wav --words audio/track.alignment.json --out talking_head

--fps (default 24, H3's rate) is the grid the cuts snap to; --min-seconds 5 and --max-seconds 14 are the chunk lengths, inside H3's 15 s.

What you get

• talking_head/chunkNN.wav: each chunk's audio, sample-exact from the track at its own rate; end to end they are the track (163.5 s at 48 kHz: 7,848,960 samples in 18 chunks, the track's exact count).
• talking_head/chunks.json: fps, and per chunk start, end, seconds, frames ([round(start·fps), round(end·fps)]), sentence_end, audio and text.
• One line a chunk on stdout: . for a cut after a sentence, ~ for one inside a sentence.

Tips and traps

• Cut where the sound pauses, not where the transcript does. On a webcam read, whisper stretched a sentence's last word back across a 0.34 s pause that came just before it, so that pause looked like the sentence's end; the envelope places the pause, and a word only marks a sentence end when it started before the pause did.
• A ~ chunk ends mid-sentence: no sentence ended 5 to 14 s after its start. The presenter returns to the pinned pose there, which reads as a hitch; look at that seam first on review, or re-voice that stretch with a pause.
• "no pause between …": a stretch with no pause of 0.25 s. Raise --max-seconds to 15 at most, or re-voice it with a pause.
• A chunk's text from a whisper transcript can gain or lose a boundary word where the transcript smears; read it against the chunk's WAV, or transcribe the WAV alone, before it goes into the prompt.
• On a 163.5 s TTS read the plan cut 18 chunks of 5.5 to 13.8 s, every cut after a sentence; the earlier plan from word gaps alone had 17 at almost the same places.

Check it

• The chunk WAVs' sample counts add up to the track's.
• Every chunk is 5 to 14 s; read the ~ lines.
• frames of chunk n+1 start where chunk n's end.