Map a song's bars

Give it the song and, for the best grid, its drum stem and the vocal's word timings; get beats.json (tempo, beats, downbeats, drum onsets, energy) and a printout of every sung phrase with the bar it starts in. Write the shot list on that grid, then cut with python-cut-to-bar-grid and time captions with python-time-lyrics, which both read beats.json.

Inputs

• The song as WAV, 44.1 kHz stereo, converted from the source (demucs-split-song-stems, Inputs). That WAV's length is the song's end for every bar past the last downbeat.
• The drum stem (demucs-split-song-stems), optional but recommended: it picks the downbeat phase.
• Word timings of the vocal stem, optional: ElevenLabs speech-to-text JSON (words with text, start, end, type) or whisper-cli JSON from whisper-cli -f x.wav -l auto -ml 1 -sow -oj -of out (one segment per word, offsets in milliseconds). The script reads either.

Run it

Write references/map_song_bars.md's script into the job's scripts/ as map_song_bars.py, unchanged, and run it with skill set so exec adds librosa:

exec("python", ["scripts/map_song_bars.py", "analysis/song.wav", "--out", "analysis/beats.json",
     "--drums", "analysis/stems/drums.wav", "--words", "analysis/vocals_stt.json"],
     cwd="<job>", skill="python-map-song-bars")

A bar as a time and a frame, for checking a shot's slot or a cue:

exec("python", ["scripts/map_song_bars.py", "analysis/song.wav", "--frames", "analysis/beats.json",
     "--at=-1,0,12,40.5", "--fps", "24"], cwd="<job>", skill="python-map-song-bars")

Write --at= with the equals sign so -1 isn't read as an option.

What you get

beats.json:

The printout, for a 3:22 pop song at 112 BPM:

tempo 112.3 BPM, 94 bars of 2.137 s, first downbeat 0.412 s, song 202.180 s
bar    3 (   6.82)    7.31  I left the porch light on for you
bar    5 (  11.10)   11.36  counting cars along the bend

Each phrase is words with less than 0.35 s between them; the bar is the one the first word falls in, counting a word up to 0.25 s early as on the bar.

Bars are numbers everywhere downstream: 0 is the first downbeat; a fraction is part of a bar (12.5 is beat 3 of bar 12 in 4/4); -1 is the song's start (0.0 s, for an intro before the first downbeat); any bar past the last downbeat is the song's end. bar_frame is round(bar_time × fps), so shots cut at consecutive bar frames add up exactly and never drift.

Tips and traps

• Set the sections yourself from the printout and the energy curve. A model that listened to the song read the singer, harmonies and big moments well, but its section labels were off by one.
• Never trust timestamps embedded in a song file. One generator's embedded lyric timings were 5 s off. The lyric sheet itself may be in the file's tags: exec("ffprobe", ["-v", "error", "-show_entries", "format_tags=lyrics", "-of", "default=nw=1", "source/song.m4a"], cwd).
• The WAV is the length to cut to. A converted WAV was 0.19 s shorter than its M4A reported; the grid ends at the WAV's length.
• The vocal often starts a beat into the bar, and a repeated section can be sung tighter than the first time: check each sung line against its shot's bars, not the section's.
• Writing the shot list: 1 to 2 bars per shot in verses and choruses, half bars for a stutter. At 112 BPM a bar is 2.14 s, a half bar 1.07 s.
• Downbeats assume 4/4 and a steady tempo. For a song in 3/4 or with a tempo change, the bar numbers drift: check them by ear at each section.

Check it

• Cut 4 s of the song from three downbeats, near the start, the middle and the end (ffmpeg-cut-audio-window), and listen: each window starts on the "one".
• Bar length times bar count roughly equals the time from the first to the last downbeat, and the printout's phrases start where you hear them.