Crop to vertical

Give it a 16:9 cut, its shot table and a window per shot; get back the 9:16 version at native crop size (608×1080 from 1080p): each shot cropped, the full vertical cut over the source audio, a small copy and a contact sheet. It is the cheap first pass of any vertical version: crop everything, watch the whole cut, and only then restage the shots a crop can't save (nano-banana-restage-for-new-aspect), whose renders splice back in here. For a 16:9 re-cut with replaced shots, use ffmpeg-splice-into-source-cut.

Inputs

The 16:9 cut: size, rate, frame count (ffprobe-inspect-media).
The shot table, first and last frame inclusive, covering the whole cut (ffmpeg-scene-cuts, or the job's existing table).
A window per shot, one of:
- X: the window's left edge, fixed (608 wide at 1080 high, so 0 ≤ X ≤ 1312);
- X0..X1: a linear pan from X0 at the shot's first frame to X1 at its last;
- fit:PAD: the whole frame scaled to 608 wide and padded top and bottom with colour PAD (fit:0xFFFEF5), for end cards and graphics wider than the window.
Optionally, picked renders of restaged shots, at least the shot's length.

Place the windows

For each shot, make a sheet of 4 frames across it with a vertical grid every 96 px and read where the subject is on each:

exec ffmpeg -v error -y -i <cut>
  -vf "select='eq(n,25)+eq(n,30)+eq(n,35)+eq(n,40)',drawgrid=w=96:h=1080:t=2:c=white@0.6,scale=960:-2,tile=4x1:padding=4"
  -frames:v 1 -update 1 <job>/A_9_16_croponly/shot02_grid.jpg

Follow the subject and the action across the whole shot, not just its first frame; pan when they travel. Write each window, the reason and the command in the Log of AGENTS.md.

Run it

Per shot, at native crop size, no audio (for checking, upscaling and editing). Shot 01, frames 0–24, window at X = 646:

exec ffmpeg -hide_banner -loglevel error -y -i <cut>
  -vf "trim=start_frame=0:end_frame=25,setpts=PTS-STARTPTS,crop=608:1080:646:0,setsar=1"
  -an -r 24 -c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p
  <job>/A_9_16_croponly/shot01.mp4

The other two windows change only the -vf. A pan from 400 to 900 over frames 25–40, and a fitted end card over frames 439–483:

trim=start_frame=25:end_frame=41,setpts=PTS-STARTPTS,crop=608:1080:'400+(900-400)*n/15':0,setsar=1
trim=start_frame=439:end_frame=484,setpts=PTS-STARTPTS,scale=608:-2:flags=lanczos,pad=608:1080:(ow-iw)/2:(oh-ih)/2:color=0xFFFEF5,setsar=1

The pan divides by LAST − FIRST (40 − 25 = 15), so it reaches X1 on the last frame.

The full cut: every shot's chain into one concat, over the source audio. A restaged render takes its first N frames (the shot's length; its tail is dropped), scaled to cover the window and centre-cropped:

exec ffmpeg -hide_banner -loglevel error -y -i <cut> -i <restage>/video/shot03_h3lipsync_d.mp4
  -filter_complex "
    [0:v]trim=start_frame=0:end_frame=25,setpts=PTS-STARTPTS,crop=608:1080:646:0,setsar=1,format=yuv420p[v0];
    [0:v]trim=start_frame=25:end_frame=41,setpts=PTS-STARTPTS,crop=608:1080:'400+(900-400)*n/15':0,setsar=1,format=yuv420p[v1];
    [1:v]trim=end_frame=33,setpts=PTS-STARTPTS,scale=608:1080:force_original_aspect_ratio=increase:flags=lanczos,crop=608:1080,setsar=1,format=yuv420p[v2];
    …one chain per shot…
    [v0][v1][v2]…concat=n=<shots>:v=1:a=0[v]"
  -map "[v]" -map 0:a -r 24
  -c:v libx264 -preset slow -crf 16 -pix_fmt yuv420p -c:a copy
  <job>/A_9_16_croponly/A_9_16_croponly_cut.mp4

Then a small copy (-vf scale=540:960:flags=lanczos -r 24 -c:v libx264 -preset veryslow -crf 28 -pix_fmt yuv420p -c:a aac -b:a 96k, named <cut>_small.mp4) and a sheet of the first, middle and last frame of every shot, two shots per row:

exec ffmpeg -v error -y -i <vertical cut>
  -vf "select='eq(n,0)+eq(n,12)+eq(n,24)+eq(n,25)+eq(n,33)+eq(n,40)+…',scale=216:384,tile=6x<rows>:padding=4"
  -frames:v 1 -update 1 <job>/A_9_16_croponly/crops_sheet.jpg

The delivery's per-shot files

A cropped shot: its crop at exactly the shot's length (the source cuts straight to the next shot, so no tail), with the source audio for that span (-af "atrim=start=FIRST/fps:end=(LAST+1)/fps,asetpts=PTS-STARTPTS", AAC 192k).
A restaged shot: its picked render with the 0.5 s tail and the render's own audio, scaled to cover 608×1080 and centre-cropped.
Each with a _noaudio copy (-an -c:v copy), and the cut beside them.

What you get

The crop-only cut to watch whole before deciding anything else, the per-shot crops, the small copy and the sheet. On the first job, 10 of 12 shots survived as crops; two were restaged.

Tips and traps

Restage only when the crop loses a story beat: a second character, an interaction, the product. Not for empty space or a slightly tight frame.
-r <fps> on every output. Without it ffmpeg 7 drops the last frame of a trimmed segment (25 in, 24 out). Never put fps= in the per-shot chains before the concat (one frame lost per shot: 472 of 484).
A window that leaves the frame fails: keep X and X + 608 inside 0–1920 at every point of a pan.
A restaged 16:9 render's window: if the centred crop holds only a sliver of someone, a push-in drifts them out. Move the window and lock the camera in the render's prompt.
Deliver at the native crop size, 608×1080. Upscaling a crop to 1080×1920 adds cost, not detail; a shot that must be full-quality 1080×1920 is restaged and rendered natively at 1080p 9:16 instead (guide-prepare-renders-for-delivery).

Check it

Every shot file and the cut have the expected frame count (exec ffmpeg -v error -i <file> -map 0:v -f framemd5 -, counting lines that don't start with #); the cut matches the source.
Look at the contact sheet, nudge windows, re-run only the shots you changed, and review the whole cut at native size before deciding what needs more than a crop.

Hypernatural

ffmpeg-crop-to-vertical