Talking head: best-takes rough cut, then an EXTRA interactive graphics pass

Two stages, each gated by the person. Stage 1 is free and takes about 10 minutes. Stage 2 is free (all local) and the first build takes about an hour. Everything runs through exec in the job folder, with scripts kept in scripts/ and titles/. Nothing in this recipe assumes a brand: the look comes entirely from the brand kit the person supplies in Stage 2.

Stage 1: rough cut of the best takes

1. Audio: extract 16 kHz mono with ffmpeg. Note the movie's audio stream start in ffprobe (screen recordings often start audio ~0.1 s in): movie time = wav time + offset.
2. Speech islands: build a 10 ms RMS envelope in Python. Threshold = max(floor + 12 dB, p95 − 30 dB). Merge gaps under 80 ms and drop blips under 60 ms.
3. Utterances: group islands at gaps of 0.45 s or more. Transcribe each utterance on its own with whisper large-v3-turbo (pass its model file with -m). Per-utterance transcripts are exact about which words; whisper-base word times on a full file can drift up to 1 s, so never cut on them.
4. Pick takes:
   • Per line, the last complete take, which is usually the cleanest and fastest.
   • Drop false starts, filler "OK"s, coughs and "Thank you" hallucinations on silence.
   • When a long take restarts mid-sentence, cut at an energy dip inside the island; find it by printing the envelope around the joint.
5. Check any mid-island cut by transcribing the window itself (raw audio, ±0.5 s padding). A clipped last word is the most common note: a short tail can be the end of the word, not the start of the next one.
6. Squeeze pauses:
   • Inside each take, every gap over 0.14 s becomes about 0.11 s (keep 0.06 s after the offset and 0.05 s before the onset).
   • Take heads get 0.05 s of pre-roll and take tails 0.09 s.
   • A take that ends right before an unwanted word gets no tail pad (end exactly at the dip).
7. Build the cut: one ffmpeg filter_complex (ffmpeg 9 uses -/filter_complex file): trim/atrim per piece, 6 ms fade-in and 10 ms fade-out, then concat. Snap every boundary to the frame grid (round(t·fps)/fps) so audio and video stay the same length.
8. Verify: transcribe the cut with large-v3-turbo and list the longest remaining pauses. Then put it on a review page.

Stage 2: the EXTRA pass

Ask for the brand kit first

Before designing anything, ask the person for their brand assets, and write the answers in brand-kit.md in the job folder:

• Fonts: files (OTF/TTF/WOFF2) or Google Fonts names for headline, caption and accent type. Copy files into titles/assets/.
• Colours: hex values for primary, accent, shadow or ink, and background, with which one is the "active word" colour.
• Logo and wordmark: PNG or SVG with transparency.
• Characters or mascots: artwork for each pose they want used, plus any rules for how a character may and may not appear.
• Style references: past videos, a brand or design-system doc, or examples they like and dislike (boxes or no boxes, outlines or flat fills, shadows, tone).
• Words and claims: product names exactly as written, and any numbers that may appear on screen (only real ones).

If they have no kit, propose a starter look from the footage (2 colours pulled from the set or wardrobe, one heavy display font) and show it before building on it. Treat their do/don't rules as hard constraints and add them to the rules section of brand-kit.md.

Inputs to build once per cut

Rebuild all of these whenever the cut changes.

• Word timings: whisper on the cut, -ml 1 -sow -oj.
• Tracking: MediaPipe Tasks in hn4a's Python, with the models downloaded into refs/models/. Use selfie_multiclass for a per-frame person matte (temporally smoothed 0.65/0.35, smoothstep, small blur), hand_landmarker for 2 hands, and face_landmarker. Store canvas-px face box, nose and palms per frame in work/track.json.
• Person cutout: VP9 yuva420p WebM made from the cut plus the matte PNGs with alphamerge, -g 12.
• Hands-only cutout: for any moment where a title sits over the face but under the hands, take the hand-landmark convex hull, dilate it 31 px, and intersect it with the person matte.
• Freeze frame: the last frame and its cutout PNG, for an end hold of about 1.4 s.

Look

Build it from the brand kit. Render titles as PNGs in the brand's headline font and colours, drawn big so image layers with fit: fill can squash and stretch through anim.box. Text layers are only for captions and counters. Prepare character art as transparent PNGs in the brand colours (and mirrored copies for direction), following the person's rules for each character.

Structure of the spec (one build script writes it)

• A cam group overscans 40 px each side and holds a shake group. Inside shake, in this order:
  1. footage
  2. behind-titles
  3. person cutout
  4. front graphics
  5. hands cutout
• Canvas coordinates map into the overscan as x·(W+80)/W.
• Captions sit in their own group that follows the chin, outside the camera.
• Camera punch-ins (scale 1.08–1.12 toward the face) and decaying random shakes fire on hits.
• Clean every key list: clamp negative times and drop duplicate times.

Interactive moves that work

• Opening word: big in the centre over the speaker. Then the first phrase stacked on one side of the head and the next phrase on the other side. Both fly out sideways before the hero word.
• Hero word alone, big, over the face, squashed to the floor by their hands:
  • Contact y = mean of the hands' lowest landmark. Use only frames from the top of the downward push on, otherwise hands resting low at the start flatten it instantly.
  • top = max(y0, contact); p = progress to the floor; q = min(1, p/0.65); h = h0(1−q) + hmin·q; w = w0(1 + 0.38q²).
  • Extrapolate the last tracked velocity when the hands blur out of tracking.
  • Exit with a splat about 0.1 s after flat. Add squash, stamp and splat SFX.
• Title glued to the side of the head (x/y keys from the face centre every frame), flying off the way the hand sweeps at the frame the palm crosses its edge.
• A title stretched between both palms: box from the left palm x to the right palm x, with height inversely scaled to keep area.
• Objects popping from each hand, then converging where the next title bursts out. A brand character can ride a palm or hop onto a title.
• Words dropping onto the head and bouncing (land y from the face-box top), and things rising out of the mouth.
• Graphics for the words: a bar chart that grows (anim.box), a progress bar with a stepped % counter, marquee bulbs blinking around the frame, comic starburst SFX words, confetti cannons in brand colours, a spinner that resolves to a check, a title sliced in half with clip-path polygons, a stamp, a take counter that ends on ∞, a real duration squashed into the real cut length.
• "Any style" line: each caption word in its own style, with an emoji-able word replaced by its symbol.

Captions

Word-by-word in the brand's caption font: the active word in the brand's accent colour bounces in, the rest in the main colour. Phrases end at sentence ends, pauses over 0.3 s, 5 words or 26 characters. Skip every span where a big title says the same words. End a phrase where the next skip span starts too, or it collides with the title.

Sound effects

hn4a has no SFX model, so synthesize them in numpy/scipy from a cue list the build script writes: pop, stamp, impact, bass drop, boing, sparkle, whooshes, comic boom/pow/zap, type, blip, record scratch, glitch, bonk, riser, tick, ding, click, rubber stretch, cannon, footsteps, snip, reel, clap, squash, shrink, blah formant, crumple. Pan by x position, soft-limit the sum, normalise, and mix at 0.6 under the voice. If the person has brand sounds or a sonic logo, use them in place of the synthesized ones.

Occlusion rule

Type behind the person must only be a little obstructed. After every build, run a checker: for each layer between the footage and the cutout, every 2 frames while visible, take its ink alpha resized to its animated box times the person matte. Worst share hidden must be ≤ 0.2. Titles that won't fit beside the head go in front, in the lower third over the body, or above the eyes over hair. Keep wide words' centres 400 px or more from the frame edges because of the overscan.

Gates

1. A live preview: a {"compose": spec} candidate plus stills at every effect. Check stills for edge clipping, whole words hidden, and every brand rule; then the person approves.
2. Render to revN/, check the frame count and a contact sheet, and deliver per the core skill (When it's done): the file in its folder.
3. For notes, prefer re-cueing from the build script over hand edits. A change to the cut means: re-cut, re-transcribe, re-track, re-make the cutouts and end frames, rebuild, then re-run the SFX and the occlusion check.