cutout-birefnet: BiRefNet general for Core AI (macOS 27)

BiRefNet general — Zheng, Gao, Fan, Liu, Laaksonen, Ouyang and Sebe, Bilateral Reference for High-Resolution Dichotomous Image Segmentation, CAAI Artificial Intelligence Research, 2024; code and weights MIT — exported to an Apple Core AI .aimodel. Give it a photo and it returns a soft alpha matte of the main subject: a product, a plant, a glass, an animal, a person — the background-removal job, on device.

It is what cutout uses for --subject and --person, through swift-object-cutout, and what design uses to lift a subject off its photo. Both download this repository by themselves the first time it is needed, pinned to one commit, every file checked against its published hash. Apple's built-in Vision matte stays as the no-download fallback.

Files

The host resizes the picture to 1024×1024 without keeping its aspect ratio (as the network was trained) and stretches the alpha back to the picture's size with bilinear sampling. ImageNet normalisation and the sigmoid are inside the asset.

What changed

Core AI has no deformable convolution, which BiRefNet's decoder uses. The export computes it from what it is: for each kernel tap, a bilinear sample of the feature map (four gathers from a zero-padded copy), scaled by the modulator, then a matrix product with that tap's weights, summed over the taps. Sample positions stay float32 in the float16 export. Against the authors' code in float32 on the CPU, the rewrite differs by at most 4.6 × 10⁻⁶ in alpha.

Fidelity

Run through Core AI on the GPU, against the authors' PyTorch model (float32, CPU) on the same 30 inputs at 1024²:

End to end (Swift, Core Graphics resizing) the float16 asset scores the same as PyTorch on the true-alpha sets below: SAD 23.40 against 23.39 on objects and 6.12 against 6.10 on animals.

Quality

Measured with the standard matting metrics against true alpha on AIM-500 (Li et al., 2021), which this checkpoint was not trained on (its training sets are DIS5K, DUTS, HRSOD, UHRSD, HRS10K, P3M-10k and a humans set). 100 objects (plants, furniture, toys, fruit, glass), 100 animals and the 100 portraits, at their own size (at most 1920 px). Lower is better except IoU; the edge band is the uncertain strip round the subject where hair and fur live.

It beat Vision by SAD on 99 of the 100 objects, all 100 animals and 99 of the 100 portraits. Vision found no subject at all in 4 of the objects (counted as empty mattes); glass is the hardest category for both (SAD 211 for Vision, 51 here). Re-solving this model's alpha from colour (the crisp edge that helps Vision's softer mask) made every set worse, so cutout passes this model's alpha through.

Speed and memory

Apple M3 Max, macOS 27, GPU: the first load compiles the model for the Mac in about 15 s and keeps a 553 MB compiled copy in ~/Library/Caches/coreai-cache; later loads are quick. About 0.9 s per 1024² inference from Python, and 0.9–1.3 s per photo through cutout end to end (decode, model, edge, PNG). A folder run of 100 photos peaked at 3.2–3.8 GB of memory, against 5.6 GB for the float32 export and up to 10 GB for PyTorch on MPS.

Use

cutout photo.jpg                 # fetches this model on first use
cutout ./shots --out ./cut       # a folder, the model loaded once
cutout model remove subject      # the model and its compiled copy

Or by hand: hf download arraypress/cutout-birefnet --local-dir models, then point $CUTOUT_SUBJECT_MODEL at the .aimodel. Requires macOS 27 (Core AI) on Apple silicon. Converted with coreai-torch 0.4.3, coreai-core 1.0.0b3, torch 2.9.0 and transformers 4.57.1.

Licence

MIT, as the original: see LICENSE (the authors' copyright) and NOTICE (what the export changed).