Animated music video

The person brings a song (and its lyric sheet if they have one), the idea for the video and the look they want. You deliver a 16:9 1080p video where drawn characters act the song out, every cut lands on the song's bar grid, and kinetic captions hit the sung syllables and ride the beat. Nobody sings on camera; for lip sync and live-action scenes, recipe-multi-style-music-video is the route.

Ask at kickoff: the aspect, whether characters lip-sync (no, here), and who sings when you can't tell from the song.

Steps

1. Map the song (demucs-split-song-stems, elevenlabs-speech-to-text on the vocal stem, python-map-song-bars): stems, sung words with times, tempo, beats, downbeats, and every sung phrase printed with its bar. That printout is the grid the shot list is written on. Set the sections yourself from it; never trust timings embedded in the song file. Under $1.
2. Treatment (a prose file in the job folder): the concept, a beat map per section (bar and time, picture, caption idea), and the two caption layers: lyrics baked into the picture (carved, painted, written on things) and kinetic type over the cut. Gate: the person agrees it before anything is generated.
3. Style frames (nano banana Pro text-to-image, nano-banana-prompt-format): four options per look. Describe the technique (linework, halftone, print texture, palette), not just a reference film, and ban what must not appear. Bake a lyric into each frame to judge how type reads in that look. Gate: a pick per look.
4. Cast the characters before any shot: four designs each as a 2x2 model sheet (nano banana Pro edit, the picked style frame as a style-only reference: "copy its drawing style only"), split with python-split-character-sheet. Remove stray lettering from the picked sheet with a boxed edit pasted back (nano-banana-targeted-fix, python-crop-and-paste-back). Redraw the picked style frames with the cast, then make text-free copies: those are the style references for every shot. Gate: a pick per character.
5. Shot list on the bar grid: one row per shot with start bar, end bar, sung words, look, picture, baked lettering and overlay idea, on a review page. One to two bars a shot in verses and choruses, half-bars for a stutter. Check each sung line against its shot's bars: the vocal often starts a beat into the bar, and a repeated section can be sung tighter. Gate: the shot list.
6. Stills, one call per shot (nano banana Pro edit, 2K, four at once is fine; when one lands, tasks with until: "any", look at it and start the next shot): the text-free style frame as Image 1 for style only, the model-sheet views of whoever is in the shot, the shot as one dynamic first frame, its light, and the exact lettering or "no text anywhere". references/shot-still-prompt.md has the prompt. Pass the cast per shot, by who is in it. Look at every still on a contact sheet and re-roll the misses before the person sees them. Gate: every still.
7. Animate each picked still (guide-animate-a-shot, h3-max-image-to-video or kling-v3-image-to-video): what the frame shows, what happens, then the shared rules (Choices). Render at least 24 frames at 24 fps, since H3 Max's floor is 0.92 s: let the cut trim. Gate: first test both models on the first minute of the song, cut on the bars, and let the person pick the model; then show the rest the same way.
8. Cut on the bar grid (python-cut-to-bar-grid): each shot gets exactly the frames between its bars, one look's shots can be stepped on twos (12 fps held twice), and the check finds every cut in the result. Gate: the cut with the song, before any captions.
9. Time the lyrics (python-time-lyrics): the lyric sheet timed word by word from the vocal STT. Check every ? word by ear, copy a clean chorus's timing onto a garbled repeat, and put fast repeated words on the beat grid instead.
10. Kinetic captions (guide-add-titles-and-motion-graphics, hn4a-compositor-in-scene-titles): a build script writes the compositor spec, the cut as the bottom layer and the song as its audio. references/kinetic-captions.md has the design. Skip lines already baked into a shot. Gate: the spec live on a review page with stills of every special moment beside it, before the render; fix a spot after the render with --frames A:B --into.
11. Deliver (guide-prepare-renders-for-delivery): the 1080p H.264 at CRF 19, the render master, and a 720p small copy (halftone texture compresses poorly: 332 MB at 1080p for 3:22).

Choices this recipe makes

• The song's bars, not seconds, are the timeline. The shot list, the cut and the captions all read the same downbeats, so nothing drifts over three minutes.
• Cast before shots. Without model sheets, designs drifted between shots.
• Text-free style references. Every shot copies its look from a style frame with no lettering in it.
• Shared rules on every animation brief: keep the first frame's drawn style (linework, halftone, print texture, colours; never photographic); nobody talks or sings, mouths closed except brief expressions; lettering already in the frame stays exactly as drawn and never morphs, and no new text appears; one continuous shot, no cuts; the characters stay on model.
• H3 Max for a whole cartoon video. On cartoon shots H3 Max and Kling v3 Pro split about evenly: H3 is better at big physical action (it commits at once), Kling better at keeping baked lettering and graphic detail intact (H3 warped a carved line once; Kling broke a word once). The person picked H3 for every shot: cheap, fast, and its motion read best cut together.

Dead ends

• Timestamps embedded by the song generator: 5 s off.
• A model's section labels for the song: off by one (its read of the singer and harmonies was right).
• Style frames with lettering as references: nano banana Pro copied the lettering into new shots, partly (half of a carved lyric line) and added sound words (HISS, SLAP!) unless told not to. Text-free references fixed both.
• Baked text written as a note: "TITLE + THE END stamp" was printed literally. Write the exact lettering.
• Kling for big cartoon action: it idles, then snaps, and drifts into odd close-ups at the end.
• Guessed type widths: pulsing words slid into each other and tilted lines ran off the frame. Measure with the same font files.

Worked example

A 3:22 pop song at 4/4, two cartoon animal characters (Character A sings, Character B is "you"), each with their own look (an ink comic with coarse halftone and misregistered plates; a xerox-and-spray-paint collage), the two looks clashing on the bridge and merging on the last chorus. 84 shots, no lip sync, about $120 of generation in all.