finegrain-box-segmenter INT8 G128 sym computeFP16 ONNX

Weight-quantized finegrain-box-segmenter-ONNX with fp16 activations, shrinking the model from 805 MB to 122.5 MB (6.6× smaller; 3.2× smaller than the parent's 396 MB build) and its GPU memory to about a quarter of the plain export's on WebGPU and DirectML (the parent runs the same low-memory graph, and against it this model saves another 10-45% of GPU memory), producing output borderline identical to to full width (check the extreme examples below)

• Base model: finegrain/finegrain-box-segmenter (MIT) → MVANet, Swin-B backbone, ~94.6 M parameters, static 1024×1024 input, batch 1.
• What this repo adds: int8 quantization of the weights of every weight-carrying MatMul (MatMulNBits, 174 weight matrices), a loss-free shrink of the graph constants, a loss-free low-memory rewrite of the graph (same outputs, peak RAM halved) and fp16 activations (GPU memory down another 10-45%) - no architecture change, no retraining, same input [1,3,1024,1024] f32 → logits [1,1,1024,1024] f32 (raw logits, apply sigmoid) contract.
• Made for GPUs - WebGPU, DirectML and CUDA, from onnxruntime-node / onnxruntime-web / Python / PHP (verified end-to-end on ORT 1.26: every node, all 654 MatMulNBits nodes included - the 174 weight matrices, the backbone's ones shared by 5 per-view passes - is placed on the WebGPU EP). It runs on CPU too, but there the fp16 activations make it 1.5-1.8× slower than the fp32 parent - for CPU-only use, take the parent.

Quantization

• INT8 - int8 weights on all 174 weight-carrying MatMuls (MatMulNBits); convs, LayerNorm and the relative-position tables are not quantized.
• G128 - group size of 128 weights, measured best for this model.
• sym - integer grid is fixed at zero - this, combined with each group of weights having its own scale is what gives the closest mask edge to the fp32 ONNX.
• computeFP16 - fp16 activations: 10-45% less GPU memory, slower on CPU (step 9 below).
• ONNX - verified with ONNX Runtime 1.23+ in Node, Python, PHP and browsers (CPU, WebGPU, DirectML, CUDA) and Transformers.js 4.0+.

Recipe

Nine steps, each checked against the fp32 reference on a fixed 11-image eval set:

1. Zero-mask strip - 404.8 MB of all-zero Swin pad masks become ConstantOfShape → −405 MB, bit-identical.
2. {-100, 0} masks → fp16 - stored fp16 with a Cast back → −9.9 MB, bit-identical.
3. MatMulNBits int8 - RTN, group 128, symmetric, all 174 weight-carrying MatMuls → 128.9 MB.
4. Rank-flatten - the quantized MatMuls get 2-D inputs → DirectML at 8k 13.9 → 0.5 s, same output.
5. Pad surgery - 24 dynamic-pad ScatterND sites become static Pad nodes, so onnxruntime-web's load-time folding fits the 4 GB wasm heap → loads in every browser with default options, bit-exact.
6. Low-memory rewrite - six exact rewrites (polyphase convs and a 16-stripe full-resolution tail, the Swin backbone per view, cross-attention in 8 token chunks, …) → peak RAM −50%, WebGPU VRAM −69%, same outputs.
7. DirectML partition cuts - 166 no-op Concat(x, Slice(x, 0:0)) pairs stop DirectML from compiling the decoder into one huge graph → DirectML VRAM 7.7 → 2.0 GB, bit-exact on CPU and WebGPU.
8. WebGPU fan-out - wide Concat/Split ops become trees of at most 4, so no op needs more than 5 storage buffers → runs on default WebGPU limits, bit-exact.
9. fp16 activations - activations, scales and conv/LayerNorm parameters in fp16; attention MatMuls (WebGPU's fp16 MatMul is too lossy) and Pad (Firefox can't compile its fp16 shader) stay fp32 → GPU memory −10 to −45%, WebGPU 1.5-2.2× faster, mask MAE 0.00099 → 0.00110, slower on CPU. Produces the shipped onnx/model.onnx.

Input and Output

It's identical to my fp32 ONNX: input input [1,3,1024,1024] float32 (ImageNet-normalized NCHW RGB), output logits [1,1,1024,1024] float32 (raw logits, apply sigmoid yourself) - also see ## Usage below and /usage folder in the repo for a fully functioning examples. Or just point an LLM here, and it will do it all for you, adjusting it to your project's tech stack.

Examples

The 8 intentionally extra-difficult tests, processed with the same production pipeline (whole-image, no boxes/prompts) on the original fp32 PyTorch safetensors model, the fp32 ONNX parent and this int8 model here. Each example has a full-size image in the first row - you can click it to open in a full resolution - and a cropped sample in the second row.

Comparative numbers

TL;DR: this int8 is a GPU model. It is 3.2× smaller than the fp32 ONNX, and on every GPU path it needs less memory - 35-45% less on WebGPU in Node and Chrome and on CUDA, ~20% less on DirectML, ~10% less in Firefox's WebGPU - at the same speed or better (1.3-1.7× faster in Firefox's WebGPU, up to 15% faster in Chrome's). On CPU it is the other way round: ONNX Runtime's CPU kernels are mostly fp32, so the fp16 activations get converted back and forth around them, and the native runtimes take 1.5-1.8× as long as with the fp32 model (at about the same RAM), the browsers' wasm 1.1-1.45× as long. For CPU-only use, take the fp32 parent.

All ONNX columns were measured on 2026-10-05 on the files both repos ship (fp32 sha256 a7db4845…, INT8 d75f5629…), each INT8 cell behind a check that no other job was loading the CPU; the fp32 Safetensors column on 2026-09-23 (CPU) and 2026-10-05 (CUDA). The INT8 browser cells ran in fresh browser profiles, the fp32 ones in everyday profiles; fp32 control cells (1080p) run in fresh profiles came out 13% faster in Chrome's wasm, 18% slower in Firefox's and 7-8% slower in both WebGPUs, so the browser comparisons between the two columns carry about that much uncertainty (in Node the same control cells match to within 1-3%). *The fp32 ONNX columns are the parent's current low-memory build: the same graph rewrites as here (recipe steps 1, 2 and 5-8, everything but the int8 quantization and the fp16 activations), so the memory savings of steps 6-7 apply to both models. For reference, the plain browser build of the export (389.5 MB, without recipe steps 6-8) peaks at 2.3 GB of RAM on CPU, 5.8 GB on WebGPU and 7.7 GB on DirectML. Node GPU memory, and all CUDA GPU memory (Safetensors included), is that process's own GPU memory (the GPU Process Memory counter); Python's DirectML and the browsers count the whole GPU over the idle level, for the browsers the highest of the four sizes, since the Windows counter under-reads single runs.

Note that on Chrome one can also run the model in WebNN, but its performance is really bad: for an 8k image it's 9.6 s @ 10.1 GB RAM vs WebGPU's 1.4 s @ 2.2 GB for the fp32 ONNX, and a 1080p image takes about as long (10.5 s @ 10.5 GB). The int8 ONNX (our quantization here, 1.3 s @ 1.3 GB on WebGPU) won't even start - onnxruntime-web's WebNN backend unpacks its 8-bit MatMulNBits weights as if they were 4-bit.

On CUDA the low-memory graph costs speed. ONNX Runtime's CUDA provider runs it node by node, so the stripes, chunks and partition cuts of recipe steps 6-7 add up: one 1024² inference takes ~0.25 s with the int8 and 0.27 s with the fp32 model, where the plain browser build of the export takes 0.18 s (at 6.5 GB) and the PyTorch original 0.19 s (at 4.6 GB). DirectML compiles the graph and loses no speed to the same rewrites. ORT's CUDA memory arena grows in doubling steps, so CUDA memory comes in steps too: the fp32 model stops at 2 GB of arena plus the ~0.3 GB CUDA context (2.4 GB), the int8 with its fp16 activations one step lower (1.35 GB). PHP's CUDA times are mostly GD resizing on the CPU (2.0 of the 2.3 s at 8k).

Tested on

• Harwdare - i7-8700K, NVIDIA RTX 5070 Ti 16 GB (driver 591.86), 32 GB DDR4 2400 MT, Windows 11
• Node - Node.js 24.18.0, onnxruntime-node 1.26.0, sharp 0.33.5
• Python - onnxruntime 1.26.0 (CPU), onnxruntime-directml 1.24.4 (DirectML), onnxruntime-gpu 1.26.0 (CUDA, with the pip CUDA 12.9 runtime and cuDNN 9.10.2), numpy 2.3.4, Pillow 12.2.0, Python 3.12.10
• PHP - onnxruntime-php 0.3.7, bundled GD, OPcache JIT off; for CUDA, the GPU onnxruntime.dll from the onnxruntime-gpu 1.26.0 wheel
• Browsers - onnxruntime-web 1.26.0, Google Chrome Dev 157.0.8081.0, Firefox Developer Edition 158.0 (INT8 columns: a fresh browser profile per cell)
• fp32 Safetensors - PyTorch 2.12.0 (+cpu, and +cu130 with CUDA 13.0 / cuDNN 9.20 for the CUDA rows), refiners 0.4.1.dev235 (Finegrain's MVANet implementation, a separate library on top of PyTorch (commit 505dbdc, latest at the time tests run)), Python 3.12.10

Usage

Every section below has a runnable counterpart in usage/, one folder per runtime: transformersjs/, web/ (a single HTML page), node/, python/ and php/. Each one removes the background of a whole image and writes <name>_cutout.png (RGBA).

Transformers.js (easiest)

The repo ships the standard onnx/model.onnx + config.json + preprocessor_config.json layout, so Transformers.js (versions ≥4.0.0 supported) can run the whole pipeline for you - pre-process, sigmoid, and resize-back included:

import { pipeline } from '@huggingface/transformers';

const segmenter = await pipeline('background-removal', 'MarcinEU/finegrain-box-segmenter-INT8-G128-sym-computeFP16-ONNX');
const out = await segmenter(['https://example.com/photo.jpg']);
out[0].save('cutout.png');     // RawImage RGBA - alpha is the matte
// out[0].toCanvas() / await out[0].toBlob() if you'd rather not save to disk

As a command-line script: usage/transformersjs/remove_bg.mjs (npm install, then node remove_bg.mjs --input photo.jpg --output photo_cutout.png; loads onnx/model.onnx of this repo, or --model <path>). All command-line runners share the same options - see Command-line options.

With the WebGPU EP every node runs on the GPU, all 654 MatMulNBits nodes included (ONNX Runtime's WebGPU EP has PRelu only for fp16, which the decoder's 11 PRelu nodes use here - see recipe step 9 - so session creation shows no "Some nodes were not assigned to the preferred execution providers" warning).

Javascript in Web Browsers (onnxruntime-web)

There is no pipeline here, so the pre/post-processing is done by hand: the whole image is squashed to 1024², run once (Swin's shifted attention windows live inside the graph, nothing to tile or slide), and the mask is scaled back - an equivalent of usage/node/remove_bg.mjs:

import * as ort from 'onnxruntime-web/webgpu';

const SIZE = 1024, MEAN = [0.485,0.456,0.406], STD = [0.229,0.224,0.225];
const session = await ort.InferenceSession.create(
  'https://huggingface.co/MarcinEU/finegrain-box-segmenter-INT8-G128-sym-computeFP16-ONNX/resolve/main/onnx/model.onnx',
  { executionProviders: ['webgpu'] });   // DEFAULT options work - see note above

// draw your image into a 1024x1024 canvas, then:
const { data } = ctx.getImageData(0, 0, SIZE, SIZE);      // RGBA, Uint8ClampedArray
const plane = SIZE * SIZE, chw = new Float32Array(3 * plane);
for (let p = 0; p < plane; p++) {
  chw[p]             = (data[p*4]   / 255 - MEAN[0]) / STD[0];
  chw[plane + p]     = (data[p*4+1] / 255 - MEAN[1]) / STD[1];
  chw[2*plane + p]   = (data[p*4+2] / 255 - MEAN[2]) / STD[2];
}
const out = await session.run({ input: new ort.Tensor('float32', chw, [1,3,SIZE,SIZE]) });
const logits = out.logits.data;                            // Float32Array, 1024*1024
const alpha = logits.map(v => 1 / (1 + Math.exp(-v)));     // sigmoid -> [0,1]
// resize `alpha` back to your image size and composite as the alpha channel.

A complete page built on this snippet (file picker, WebGPU or wasm, separate Load model / Process image buttons, a progress bar, cutout over a checkerboard, cutout and mask downloads) is usage/web/index.html. onnxruntime-web runs in a Web Worker there, so the page stays responsive; the progress is counted, not estimated - finished GPU kernels on WebGPU, ORT's per-node profiler timestamps on wasm, against the totals of a warm-up run at load - so it tracks the share of the graph done, and the last few % (the heavy full-resolution layers) take longest. On wasm the profiler keeps its events until the session ends and the 4 GB wasm heap only fits ~2 profiled runs, so the page gives each wasm session one image and builds the next one in the background (~2 s, from the model bytes it keeps). Run python usage/web/serve.py (if you won't run the server, you will get "error: NetworkError when attempting to fetch resource" caused by CORS) and open http://localhost:8000/usage/web/. Any static server works, but this one sends the COOP/COEP headers that let wasm use every CPU core. Without them the wasm path runs single-threaded and is several times slower; WebGPU is unaffected.

Same snippet as the parent card - only the model URL changed, because the I/O contract is identical. The com.microsoft opset import for MatMulNBits is understood by onnxruntime-web out of the box (the bundled wasm and WebGPU backends both implement the op; the WebGPU placement is verified via onnxruntime-node, see the onnxruntime-node section below). Pre-fetching 122 MB instead of 396 MB (805 MB for the plain fp32 export) is exactly the point of this repo in a browser context.

Javascript in Node .js (onnxruntime-node, requires Sharp)

A complete, runnable background remover is in usage/node/remove_bg.mjs (with onnx/model.onnx downloaded):

cd usage/node
npm install                                                                         # onnxruntime-node + sharp
node remove_bg.mjs --input photo.jpg --output photo_cutout.png                      # CPU
node remove_bg.mjs --input photo.jpg --output photo_cutout.png --ep webgpu          # GPU (recommended)
node remove_bg.mjs --input photo.jpg --output photo_mask.png --mask                 # grayscale mask instead

It resizes the whole image to 1024², normalizes, runs the session, applies sigmoid, resizes the mask back, and writes either the --output cutout (RGBA) or, with --mask, the 8-bit grayscale mask. Outputs carry real transparency - view them over a checkerboard/solid background.

Python (onnxruntime)

Verified against this exact int8 export (pip install onnxruntime pillow numpy huggingface_hub); the same pre/post-processing as the Node reference pipeline - masks are indistinguishable between the two models on the eval set:

from huggingface_hub import hf_hub_download
from PIL import Image
import numpy as np, onnxruntime as ort

model_path = hf_hub_download("MarcinEU/finegrain-box-segmenter-INT8-G128-sym-computeFP16-ONNX", "onnx/model.onnx")
session = ort.InferenceSession(model_path)

img = Image.open("photo.jpg").convert("RGB")
x = np.asarray(img.resize((1024, 1024), Image.BILINEAR), dtype=np.float32) / 255.0
x = ((x - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]).transpose(2, 0, 1)[None].astype(np.float32)

logits = session.run(None, {"input": x})[0][0, 0]
alpha = 1.0 / (1.0 + np.exp(-logits))                       # sigmoid -> [0,1]
mask = Image.fromarray((alpha * 255).astype(np.uint8)).resize(img.size, Image.BILINEAR)

cutout = img.copy(); cutout.putalpha(mask)
cutout.save("cutout.png"); mask.save("mask.png")

For NVIDIA CUDA, pip install onnxruntime-gpu[cuda,cudnn], call ort.preload_dlls() before creating the session (it loads the CUDA and cuDNN DLLs those extras install) and pass providers=["CUDAExecutionProvider", "CPUExecutionProvider"]; the quantized graph needs nothing else - the int8 MatMulNBits op ships with every ORT build. One catch: the newest cuDNN on pip (9.27) made every convolution of ORT 1.26 fail here with CUDNN_BACKEND_API_FAILED, and ORT then quietly re-runs the model on the CPU; 9.10.2 works (pip install nvidia-cudnn-cu12==9.10.2.21).

The same code as a command-line script is usage/python/remove_bg.py: pip install -r requirements.txt (Python 3.10+), then python remove_bg.py --input photo.jpg --output photo_cutout.png (--model for a model outside this repo, --ep dml / --ep cuda with the matching ORT package, --mask to save the grayscale mask instead).

Command-line options

The Python, Node, Transformers.js and PHP scripts in usage/ take the same options (--key value or --key=value):

Before loading the model they check that the input and model are readable files and that the output is a .png in an existing, writable directory, and exit with an error otherwise. Minimum versions: Python 3.10, Node 18.17 (Transformers.js: Node 20.9), PHP 8.2. The browser page (usage/web/) offers both downloads: the transparent cutout and the grayscale mask.

PHP (onnxruntime-php)

usage/php/remove_bg.php

cd usage/php
php -d extension=ffi -d extension=gd path/to/composer.phar install   # ankane/onnxruntime-php + the ORT library (~16 MB DLL)
php -c php.ini remove_bg.php --input photo.jpg --output photo_cutout.png            # RGBA cutout
php -c php.ini remove_bg.php --input photo.jpg --output photo_mask.png --mask       # grayscale mask
# without the ini file: php -d extension=ffi -d ffi.enable=1 -d extension=gd remove_bg.php ...

It writes the cutout (RGBA, the mask as full 8-bit alpha) or, with --mask, the 8-bit grayscale mask on its own. Requires PHP 8.2+. Verified on Windows with XAMPP's PHP 8.3 against both this model and the fp32 parent: the network output is the same as onnxruntime-node's (fed the same 1024² tensor, the masks differ by at most 1/255), and what does differ comes from resizing - GD's Catmull-Rom filter is the closest match to sharp's, leaving 0.46 % of pixels more than 1/255 apart on a 1080p photo (IoU@0.5 0.997), all along edges.

A few things worth knowing:

• CPU, or CUDA with a GPU build of ORT. onnxruntime-php downloads the CPU build of ORT, and that is what the script uses. The binding also wires up CUDA (and CoreML on macOS), but has no DirectML or WebGPU path. For the table's PHP CUDA column, OnnxRuntime\FFI::$lib pointed at the onnxruntime.dll inside the onnxruntime-gpu 1.26.0 wheel, its pip CUDA / cuDNN DLLs were loaded first (what ort.preload_dlls() does in Python), and the session got providers: ['CUDAExecutionProvider', 'CPUExecutionProvider']. GD's resizes still run on the CPU, and at 8k they take 2.0 of the 2.3 s. On CPU, PHP is about as fast as Node, up to 1.3 s slower per image at 8k (see the table above).
• Don't feed it plain PHP arrays. run() with a nested [1][3][1024][1024] array, and toObject() on the output, convert element by element and cost seconds. The script allocates the tensor inside ORT, copies the packed float32 bytes in with FFI::memcpy, and reads the logits back with one FFI::string.
• Set memory_limit. PHP's default 128M is not enough; the script sets -1 itself.
• Memory grows after the first image. On CPU the second inference in the same process adds ~1 GB of RAM and then stays level (Node and Python do the same), so for batch jobs plan on ~2.4 GB rather than the peak in the table. It is ONNX Runtime's memory pattern (the 2nd run allocates its traced layout next to the 1st run's arena), so enableMemPattern: false in the session options (Node/PHP; Python: enable_mem_pattern = False) keeps it near the first-image level at no speed cost - measured in Node: 2.42 → 1.50 GB over 3 images.

References

Model

• Yu Q., Zhao X., Pang Y., Zhang L., Lu H., Multi-view Aggregation Network for Dichotomous Image Segmentation, arXiv:2404.07445, 2024

Quantization

• Microsoft, ONNX Runtime Contrib Operators - MatMulNBits, GitHub documentation - the blockwise weight-only op deployed here (int8, group 128, symmetric, per-block scales - fp16, like the activations; MatMul nodes only - Convs, LayerNorm and the relative-position tables are not quantized).
  • Microsoft, matmul_nbits_quantizer.py, ONNX Runtime 1.26 in-tree quantizer - the RTN (round-to-nearest) configuration used to produce this model. The graph surgeries (all-zero pad-mask removal, {-100,0} masks stored as fp16) are described in the recipe section above and carry no external paper.

Evaluation metrics

Used for SOD-standard set, reference-fidelity vs the fp32 model

• Fan D.-P., Cheng M.-M., Liu Y., Li T., Borji A., Structure-measure: A New Way to Evaluate Foreground Maps, arXiv:1708.00786, ICCV 2017 - S-measure.
• Lartpang, PySODMetrics: A Simple and Efficient Implementation of Grayscale/Binary Segmentation Metrics, GitHub, 2020 - reference implementation behind our metric port, MAE / IoU / boundary-F are the same SOD-standard set the MVANet paper reports.

License & attribution

MIT, inherited from the base model. All credit for the weights and architecture goes to Finegrain (finegrain/finegrain-box-segmenter) and the MVANet authors. This repository is a quantized derivative of the fp32 ONNX conversion published at MarcinEU/finegrain-box-segmenter-ONNX (same terms as the parent card); no architecture changes, no retraining - weights only.