Log in
TechniqueBy HypernaturalVersion 4, updated

BiRefNet matte

hn/birefnet-matte

Cut the people (and the podium, chair or desk in front of the back wall) out of a span of video as a foreground cutout with a real alpha channel, a VP9 WebM that lines up frame for frame with the source: a frame-exact segment, BiRefNet v2 video on fal for the mask, and an ffmpeg alpha merge. Use it to put titles behind people, or a presenter over graphics; never colour-key instead.

SKILL.md

Give it a video and a span [start, end) in seconds; get back that span's foreground as a VP9 WebM with alpha (yuva420p), short-GOP so the compositor seeks it fast. Frame k of the cutout is frame start × fps + k of the source, so laid over the source in a compose spec it covers the type behind the people exactly (SSIM 0.98 against the source on a real job, no ghost edges). Three steps: cut the segment, matte it on fal, merge the mask in as alpha. Use the cutout from hn4a-compositor-in-scene-titles (type behind people) or hn4a-compositor-motion-graphics (a presenter cut out over a slide). For one person to camera with nothing in front of them (a presenter, an avatar take, a webcam read), python-matte-a-talking-head does it free on this computer; for a matte with the face and hands tracked, python-track-face-and-hands.

Inputs

  • The video (the finished cut, or the presenter footage) and its frame rate (ffprobe-inspect-media).
  • The span, snapped to the shot's cut times (ffmpeg-scene-cuts, with the low threshold that finds dissolves), so the cutout starts and ends with the shot.

Run it

1. The segment, frame-exact at the source's own rate, near-lossless, no audio. Seek before the input; -r is the source's rate. For 0 to 9.0833 s of a 24 fps cut:

exec ffmpeg -v error -y -ss 0 -i <job>/cut.mp4 -t 9.0833 -an
  -c:v libx264 -crf 12 -pix_fmt yuv420p -r 24
  <job>/mattes/A_cutout_src.mp4

2. The mask. Call describe("fal/fal-ai/birefnet/v2/video") before the first run in a job and check these fields against it; they are the ones proven on real jobs.

run(
  model="fal/fal-ai/birefnet/v2/video",
  input={
    "video_url": "<job>/mattes/A_cutout_src.mp4",
    "model": "General Use (Heavy)",
    "operating_resolution": "2048x2048",
    "output_mask": true,
    "refine_foreground": true,
    "video_quality": "maximum",
    "video_output_type": "X264 (.mp4)"
  },
  into="<job>/mattes",
  unit="A_cutout_mask"
)

The output's mask_video is the mask (white = foreground). Models on offer: General Use (Heavy), General Use (Light), General Use (Light 2K), General Use (Dynamic), Matting, Portrait; resolutions 1024x1024, 2048x2048, 2304x2304. Heavy at 2048 is the proven setting: it keeps furniture in front of the back wall (a podium, a chair), which is right when the type sits on that wall.

3. The alpha merge: the segment plus the grey mask, a slight feather on the edge (Gaussian sigma 0.8; 0 for none), VP9 with a keyframe every 12 frames:

exec ffmpeg -v error -y
  -i <job>/mattes/A_cutout_src.mp4 -i <job>/mattes/A_cutout_mask.mp4
  -filter_complex "[1:v]format=gray,gblur=sigma=0.8[m];[0:v][m]alphamerge,format=yuva420p"
  -c:v libvpx-vp9 -b:v 0 -crf 18 -g 12 -auto-alt-ref 0 -row-mt 1
  <job>/mattes/A_cutout.webm

Keep -auto-alt-ref 0 as written (the proven VP9 alpha encode); -g 12 keeps seeks short.

What you get

A_cutout.webm, plus the segment (A_cutout_src.mp4) and the mask (A_cutout_mask.mp4) beside it. Keep all three: re-merging with another feather is a local ffmpeg run, while a new mask is another paid run.

Tips and traps

  • Never colour-key the source for a cutout: it eats skin and hair edges and leaves a visible fringe. Matte it, or keep the footage in a window where its own background shows.
  • ffmpeg's native VP9 decoder drops alpha. Put -c:v libvpx-vp9 before -i in any command that reads the WebM. ffprobe reports such a file as yuv420p with the tag alpha_mode=1; that is a cutout with alpha, not a broken one.
  • One cutout per shot that needs one, cut at that shot's boundaries and placed in the comp at the same start.

Check it

  • Frame count of the WebM equals the segment's (count framemd5 lines, decoding with libvpx): exec ffmpeg -v error -c:v libvpx-vp9 -i <cutout.webm> -map 0:v -f framemd5 -
  • Edges: one frame over a flat colour, looked at for fringes and missing hair:
exec ffmpeg -v error -y -c:v libvpx-vp9 -i <job>/mattes/A_cutout.webm
  -filter_complex "color=c=0xB8E62E:s=1920x1080:r=24[bg];[bg][0:v]overlay=shortest=1,select=eq(n\,48)"
  -frames:v 1 -update 1 <job>/mattes/A_cutout_check.png
  • Alignment: the cutout over its own segment should be near identical to the segment (SSIM about 0.98 or better, read from the log): exec ffmpeg -i <segment> -c:v libvpx-vp9 -i <cutout.webm> -lavfi "[0:v][1:v]overlay=shortest=1[o];[o][0:v]ssim" -f null -

FAQs

Cut the people (and the podium, chair or desk in front of the back wall) out of a span of video as a foreground cutout with a real alpha channel, a VP9 WebM that lines up frame for frame with the source: a frame-exact segment, BiRefNet v2 video on fal for the mask, and an ffmpeg alpha merge. Use it to put titles behind people, or a presenter over graphics; never colour-key instead.

Install Hypernatural Desktop, a free Mac app, and connect it to Claude Code, Codex or Claude Desktop. Then describe the video you want and your agent finds the skill itself, or paste this page's link (https://hypernatural.ai/skill/hn/birefnet-matte) and it installs hn/birefnet-matte and follows it.

It runs fal/fal-ai/birefnet/v2/video. Hypernatural Desktop has every one of them built in, so there are no API keys to set up, though you can bring your own.

The skill is free. You only pay for the AI generations it runs, and your agent shows you the price of every batch before it spends anything. Editing, captions and titles run on your Mac at no cost.

Yes. Ask your agent to adjust it for your job, or to save its own version. You can publish your version under your own handle so anyone can install it.

Go make something cool.

Install the app, open your agent, describe the video. That's the whole onboarding.

Sticker HypernaturalSticker BannerSticker TVBoy