Give it a video of one person speaking to camera; get their cutout, <name>_cutout.webm
(VP9, yuva420p, short GOP so the compositor seeks it fast), frame k of which is frame
k of the source, and the grey mask, <name>_mask.mp4. It runs Robust Video Matting on
this computer (Apple's GPU when there is one), free. The siblings:
| Use | When |
|---|---|
| This | One person to camera on a plain or soft background: a presenter, an avatar take, a webcam read |
birefnet-matte (paid) | The cutout must keep a podium, chair or desk, or the shot is busy; type behind people in a scene |
python-track-face-and-hands | The graphics react to the face and hands too: it tracks them and makes a matte in the same pass |
Inputs
- The video, the exact take that goes in the cut (an assembled talking head, or a
webcam rough cut), at its own frame rate and size (
ffprobe-inspect-media). - Matte the finished take, not its source chunks: one pass, one alpha that runs across the seams.
Run it
Write references/rvm_matte.md's script into the job's scripts/ as rvm_matte.py, as
is, and run it with cwd the job folder:
exec python scripts/rvm_matte.py talking_head/talking_head.mp4 mattes/presenter
--ratio is the share of the frame the model looks at for the person's outline:
0.25 for 1080p and up, 0.4 below (the defaults); raise it toward 0.6 for a person small
in the frame. The first run downloads the weights (about 15 MB) into models/.
What you get
mattes/presenter_cutout.webm (the source's own pixels with RVM's alpha) and
mattes/presenter_mask.mp4 (white = the person). Keep the mask: re-merging it with a
feather is the same ffmpeg merge as birefnet-matte, step 3.
Tips and traps
- ffmpeg's native VP9 decoder drops alpha. Put
-c:v libvpx-vp9before-iin any command that reads the WebM;ffprobeshowsyuv420pwithalpha_mode=1, which is a cutout with alpha. - It keeps the person, not what they hold or lean on: a podium or a chair edge goes
with the background. That is
birefnet-matte's job. - The first frames settle: RVM carries state from frame to frame, so its first few frames can be soft. Matte from the shot's start, never from a seek into it.
- Never colour-key a presenter instead: it eats skin and hair edges.
Check it
- Frame count equals the source's (decode with libvpx):
exec ffmpeg -v error -c:v libvpx-vp9 -i mattes/presenter_cutout.webm -map 0:v -f framemd5 - - One frame over a flat colour, looked at for fringes and missing hair (
birefnet-matte, Check it, the same command). - The cutout over its own source is near identical to it (SSIM about 0.99 or better).
