Find speech pauses

Give it a recording's WAV; get the pauses in it, read off the sound, with their length and how quiet they go, or the speech split into utterances, each cut to its own WAV. Word times from speech-to-text smear across long silences and drift on long files (whisper-cli-transcribe-with-word-times); the envelope doesn't, so cut on it. For pauses between words of a finished transcript (and lip-sync chunks), python-read-word-timings. For a whole rough cut of best takes, python-cut-best-takes-by-speech-islands, which runs on this.

Inputs

• The 16 kHz mono WAV of the recording, the same file you transcribe, so the times agree. A movie's time is the WAV's time plus the audio stream's start.
• Optionally a window (--start, --end) to look at one joint closely.

Run it

Write references/speech_map.md's script into the job's scripts/ as speech_map.py, as is, and run it with cwd the job folder:

exec python scripts/speech_map.py pauses work/voice.wav --min-gap 0.3
exec python scripts/speech_map.py runs   work/voice.wav --gap 0.45 --out-dir work/runs
exec python scripts/speech_map.py map    work/voice.wav --start 60.6 --end 62.2 --min-gap 0.02 --db -30
exec python scripts/speech_map.py snap   work/voice.wav --piece 9.2 10.9

• runs: speech islands (above the threshold, gaps under 80 ms joined, blips under 60 ms dropped) grouped into utterances at gaps of 0.45 s, padded 0.05 s, each written as run_NNN.wav. Transcribe each on its own: a short file's transcript is exact about which words were said.
• pauses: every gap of --min-gap or more, with at (its midpoint) and depth.
• map: the quiet runs inside a window, down to 40 ms, to find where to cut inside speech. A fixed --db a little under the speech level (−30 to −40) shows the dips between words that run together.
• snap: a picked piece moved so it starts 0.05 s before the speech after the pause at its start and ends 0.09 s after the speech before the pause at its end.

The threshold adapts to the recording: 12 dB over the room's floor, but never more than 30 dB under the loud speech. --db fixes it instead.

What you get

JSON on stdout (map prints a table): times in the WAV's seconds, and with runs --out-dir one WAV per utterance with its start in the list.

Tips and traps

• "Thank you" on silence. whisper writes it, or "Yes.", for a run that is only breath or room: 5 of 23 runs of a 70 s webcam read came back as "Thank you." Treat a run whose whole transcript is that as noise unless you hear it.
• Two words with no gap between them ("thinking, but") don't show at the adaptive threshold. map with --db -30 found a 0.21 s dip between them, and a cut there removed the "but" cleanly; check such a cut by transcribing the window (python-cut-best-takes-by-speech-islands).
• A window's threshold is the window's own. map on 10 s found a gap edge 0.3 s from where runs on the whole file put it. Compare like with like, or pass --db.
• Noise-suppressed webcams go to digital silence (about −81 dB) between phrases, so the floor percentile is very low and the 30 dB cap under the speech is what decides.
• Soundbites cut for a promo end in their natural pause: snap the piece, then level it with one fixed gain (ffmpeg-normalize-loudness, Tips and traps), never a per-clip loudness pass.

Check it

• Transcribe two or three runs WAVs and read them against what the person said.
• After cutting, run pauses on the cut and look at the longest that remain.