Turn a script.json and a cast (a voice and a performance guide per speaking
character) into one dialogue track, with every line's span on it. The audio comes
first: lip sync, scene renders and title cues are all timed from this track.
First question: is there more than one speaker in a scene? Then perform each scene
in one take so the turns play off each other. A narrator over a short (an ad, an
explainer, a news short) is cast and voiced in one step: the whole script, auditioned,
is the take. A presenter across a long piece is read passage by passage. If a
character has no voice yet, cast it first (guide-cast-a-voice); if there is no
script.json, make one (guide-make-a-script-from-source).
Routes
| Route | When | Quality | Leaf |
|---|---|---|---|
| TTS, one voice at a time (eleven_v4) | One presenter or narrator; single lines; auditions. The voice is a library voice or the person's own clone (elevenlabs-clone-a-voice) | One voice per call; tags direct it | elevenlabs-tts-v4 |
| A narrator for a short: the whole script, auditioned | One narrator over pictures cut to the voice, up to about 2 min (ads, explainers, news shorts) | Three whole-script takes at once; the pick is the take, and its alignment gives every word's time free | elevenlabs-tts-v4 (with-timestamps), python-read-word-timings |
| Text-to-dialogue, a scene in one take (eleven_v4) | Two or more speakers in a scene | Turns react to each other; each line's span returned; pauses approximate | elevenlabs-text-to-dialogue |
| The person's own recordings | They bring the audio of each line | As recorded; timings come from speech-to-text, like any recorded track | elevenlabs-speech-to-text, python-read-word-timings |
| Assemble the passages into one track | After TTS or text-to-dialogue | Every line and shot timed (track.json) | python-assemble-dialogue-track |
| Normalize loudness | Once, on the assembled track | −16 LUFS, two-pass linear; timings unchanged | ffmpeg-normalize-loudness |
| Transcribe the track | Every take | Words, speech rate per character, pauses | elevenlabs-speech-to-text, python-read-word-timings |
Cost is per character of text spoken; a fix re-speaks only the passages it touches. On v4, takes usually come back word-perfect (not always: transcribe each), at about 2.9 words/s for both presenters with no speed-up.
Process
- Cast arguments. Each speaking character gets its voice id and guide (the casting
voice.md, or a few sentences for a given voice). Director notes that override the guide go at the top. Record them where you keep the job's notes so every take runs with the same ones. - Group the script into passages: a scene each, merged where a line runs across a cut.
- Direct. Write each line's audio tags from its parenthetical, emotion, the beats
around it and the guide (
[warmly], and free-form ones like[lower, knowing]). The words must stay the script's: check them word for word before speaking. - Perform every passage at once, one run each with
wait: false. Follow them withtasks(wait: true, calls: [...], until: "any")and start each take's check (step 5) as it lands, calling again with the ids still running. Once every take is in, assemble: each passage is preceded by the script's silence before it, at least 0.4 s and at most 2.5 s. Plan silence into the track wherever the picture needs action time (an exit, a transition) before any video is rendered. - Check every take by transcribing it: words, words per second per character, the pauses the notes mention.
- Go on, no gate. The person approved each voice when they picked it at casting and ElevenLabs takes are reliable, so a take that passes the check goes straight to the next step; don't ask about the voice or the track again. The person hears it with the pictures, and notes they give then are fixed below. Keep two presenters in two folders (two variants); take letters are never reused.
A narrator for a short
- Ask the delivery before casting: for social, a quick conversational creator read beats a calm anchor; for an ad, "upbeat, conversational, not salesy".
- Cast three voices verified for eleven_v4 (
guide-cast-a-voice, steps 1–3), or the person's clone. - Audition each on the whole script, all three at once, on
elevenlabs/text-to-speech/with-timestamps,pcm_48000, one fixed seed, the approved text with its tags and respellings. The script is short and the read's length drives the layout, so the person hears the pacing, not just the timbre. - Check each take, free: its length and words per second
(
python-read-word-timings), every word and no tag read aloud (transcribe it,whisper-cli-transcribe-with-word-times, Check it), and its top end (Audio quality). - Gate:
review_choosewith the three takes. If the person delegates the pick, take the fastest clean read. The picked audition is the take: no second read, no paid transcript; its alignment gives the word times (python-read-word-timings).
One take, or one file per line
Never cut inside a take: a cut inside one narration take left an audible artifact the person heard on delivery. So:
- One whole take when the picture is cut to the voice (the shots start on the narration's words): shorts, explainers, most ads. Use it unedited.
- One file per line when lines must move on their own: shots of fixed length the
narration is placed onto, or a line re-voiced after a note. Voice each line on its
own, same voice, seed and tags, with the previous line as context where the model
takes it (
describe), and place each file at its cue.
Fixes, in this order
- Notes on specific lines: write those lines' tags by hand into the new take and re-speak only their passages; copy the rest. Re-prompting the LLM loses the notes.
- A pause too long or short: pause tags land only roughly. Measure it from the transcript, then insert or cut silence midway between the two words; every later line moves, so re-transcribe.
- A voice that reads too slowly: speed that character's lines up with pitch kept (atempo, at most 1.2). One v3 voice read 2.4 words/s however it was directed.
- More expression: stability 0 (the most expressive setting) on that passage only.
Audio quality
- Ask for
pcm_48000(lossless, written as WAV) instead of the defaultmp3_44100_128, and normalize the assembled track to −16 LUFS, keeping the raw one. Normalize after assembling, never per line. - A voice that sounds "like a telephone" is usually band-limited at the source: measure
the library preview's spectrum before casting (Top end, below; Voice A had nothing above ~12 kHz and
99% of its energy under 5 kHz). No output format adds what isn't there, and
denoisers (
veed/clean-audio,deepfilternet3) don't help. - Optional: when a voice sounds a little tinny, an EQ plus ffmpeg
aexciter(a synthesized top end) can make it a bit more vibrant. No fixed settings: try two or three and put them to the person withreview_choose, the plain track as one more option, and let them pick by ear (elevenlabs-tts-v4). - Pronunciation. Respell what the voice misreads in the text you send, never in the approved script, and check it in the transcript. Initials: one job's "H.R." still read as "H&R" and "H-R" fixed it, while another's initials read right with their periods, so try and check. Brand names with odd capitals as spoken ("eVolt" as "ee-volt"), README as "read-me", tool names phonetically ("F-F-mpeg"), decades and years as words ("twenty-thirties"), version numbers and prices as spoken. Captions map back to the script's spelling, not the respelling.
Top end
A voice with little above 8 kHz sounds dull on phones. Compare the takes before the
pick with top_end.py (write references/top_end.md's script into the job's
scripts/ unchanged; it needs ffmpeg and numpy):
exec("python", ["scripts/top_end.py", "audio/auditions/voice_a.wav", "audio/auditions/voice_b.wav", "audio/auditions/voice_c.wav"], cwd="<job>")
Three whole-script auditions on one job measured −25.5, −23.2 and −19.6 dB above 8 kHz; the same brightest take low-passed at 7 kHz read −33 dB. A take several dB below the others is the dull one; say so when you put the three to the person.
Avoid
- The
speedvoice setting: v3 and v4 ignore it (same length at 1.0 and 1.2). previous_text/future_texton v3: rejected.- Library professional clones not trained for v4: they render but can drift. Prefer voices verified for eleven_v4.
- Re-prompting the LLM to fix a director's line notes: hand-write the tags.
- Laying this track under clips a video model re-voiced: see
guide-lipsync-video.
Related
guide-cast-a-voice,guide-make-a-script-from-source: before this.guide-lipsync-video,guide-add-titles-and-motion-graphics: they readtrack.jsonand the transcript's word times; re-cue from the new transcript after every re-take.


