MultiMatte

Cut anything you can name.

MultiMatte is a promptable matting model: name an object and it returns an alpha matte for it. It is a LoRA fine-tune of SAM 3, retrained to produce continuous opacity instead of binary masks, with the adapter merged into the released weights.

Prompt steering

The same photograph, four prompts.

Plural concepts return every match in one matte, and small objects stay addressable.

The output is a continuous alpha matte, not a threshold, so edges hold up at 1:1 zoom.

Installation

pip install nobg

Usage

from nobg import AutoModel, AutoProcessor

model = AutoModel.from_pretrained("feyninc/multimatte")
processor = AutoProcessor.from_pretrained("feyninc/multimatte")

# Prompt-free: uses the processor's default_prompt ("the main foreground subject").
model.predict(processor, "photo.jpg").save("output.png")

# Named concept.
model.predict(processor, "photo.jpg", "the dog").save("dog.png")

predict runs the whole pipeline — load, preprocess, forward under no_grad in eval mode, post-process, composite — and returns an RGBA cutout at the input's original resolution. image accepts anything loadimg takes: a path, URL, base64 string, numpy array or PIL image.

The signature is predict(processor, image, prompt, boxes), everything optional after image. Useful keywords: batch_size (images per forward pass, default 1 to keep peak memory flat) and return_type="alpha" for the raw (H, W) matte tensor instead of a cutout.

Results

S-measure (S_α), prompt-free, higher is better. Both columns come from one scoring harness on identical rows, so the difference isolates the weights — the SAM 3 numbers are a fresh rescore, not values copied from a paper. Changes below 0.002 S_α are treated as measurement noise.

† No sibling in the training mix. DAVIS-S and DUT-OMRON are the two fully cross-domain splits here, so they are the pair to read for generalization — and MultiMatte's best absolute score lands on DAVIS-S at 0.979.

Naming the concept helps, before and after training. On DIS-VD, a real human-written phrase adds 0.150 S_α to base SAM 3 for zero gradient steps, and still adds 0.036 to MultiMatte after fine-tuning. Prompt supervision made the model better at both pathways rather than making it prompt-insensitive.

Training

Citation

@note{multimatte2026,
  title  = {MultiMatte: Cut Out Anything You Can Name},
  author = {Hichri, Hafedh and Feyn Research},
  year   = {2026},
  venue  = {Feyn Field Notes}
}

Please also cite the base model and the adaptation method:

@article{sam3,
  title={SAM 3: Segment Anything with Concepts},
  author={Carion, Nicolas and Gustafson, Laura and Hu, Yuan-Ting and Debnath, Shoubhik and Hu, Ronghang and Suris, Didac and Ryali, Chaitanya and Alwala, Kalyan Vasudev and Khedr, Haitham and Huang, Andrew and Lei, Jie and Ma, Tengyu and Guo, Baishan and Marks, Markus and Greer, Joseph and Wang, Meng and Sun, Peize and R{\"a}dle, Roman and Afouras, Triantafyllos and Mavroudi, Effrosyni and Dollar, Piotr and Ravi, Nikhila and Saenko, Kate and Zhang, Pengchuan and Feichtenhofer, Christoph},
  journal={arXiv preprint arXiv:2511.16719},
  year={2025},
  url={https://ai.meta.com/research/publications/sam-3-segment-anything-with-concepts/},
}

@article{lora,
  title={LoRA: Low-Rank Adaptation of Large Language Models},
  author={Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu},
  journal={arXiv preprint arXiv:2106.09685},
  year={2021},
}

Acknowledgements

Built on Meta's SAM 3. FlowDIS supplied the human-written DIS5K phrases used for training and evaluation. Thinking Machines' LoRA analysis informed the adapter configuration. Thanks to the dataset authors whose released work made the training mix and evaluation possible.