FeyNoBg — ONNX CPU Inference Benchmark

Optimized ONNX export of FeyNoBg (BiRefNet-based background removal model), benchmarked exhaustively on CPU with FP32, FP16, and Dynamic INT8 quantization.

---

What's in this repo?

---

🎨 Inference Examples

---

🚀 Quick Start

Install dependencies

git clone https://huggingface.co/mmahdi-sz/FeyNobg-ONNX
cd FeyNobg-ONNX
pip install onnxruntime numpy pillow

Run inference (ONNX — recommended for CPU deployment)

import numpy as np
import onnxruntime as ort
from PIL import Image

# ── Load model ──────────────────────────────────────────────────────────────
# Recommended: Dynamic INT8 + 8 cores for the best latency/cost ratio
ONNX_PATH = "feynobg_int8_dynamic.onnx"
NUM_CORES  = 8   # adjust to your server's core count

sess_opts = ort.SessionOptions()
sess_opts.intra_op_num_threads = NUM_CORES
sess_opts.inter_op_num_threads = 1
sess_opts.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_ALL

session = ort.InferenceSession(
    ONNX_PATH,
    sess_options=sess_opts,
    providers=["CPUExecutionProvider"]
)

# ── Preprocess ──────────────────────────────────────────────────────────────
def preprocess(image_path: str, size=(1024, 1024)) -> np.ndarray:
    img = Image.open(image_path).convert("RGB").resize(size)
    x   = np.array(img, dtype=np.float32) / 255.0
    mean = np.array([0.485, 0.456, 0.406])
    std  = np.array([0.229, 0.224, 0.225])
    x    = (x - mean) / std
    return x.transpose(2, 0, 1)[np.newaxis].astype(np.float32)  # [1, 3, H, W]

# ── Inference ───────────────────────────────────────────────────────────────
input_tensor = preprocess("your_image.jpg")
input_name   = session.get_inputs()[0].name

alpha_mask = session.run(None, {input_name: input_tensor})[0]  # [1, 1, H, W]
mask = (alpha_mask[0, 0] * 255).clip(0, 255).astype(np.uint8)
Image.fromarray(mask).save("output_mask.png")
print("Done! Mask saved to output_mask.png")

Run inference (PyTorch — original model)

pip install nobg torch

import nobg
import torch
from PIL import Image

model = nobg.AutoModel.from_pretrained("nobg/FeyNobg")
model.eval()

# ... standard PyTorch inference

Note: PyTorch (16 cores, no pinning) averages 40,743ms per image. The ONNX INT8 model on 8 pinned cores achieves 11,165ms — 3.6× faster.

---

📊 CPU Benchmark Results

All benchmarks run on an AMD EPYC-Genoa Processor (16 cores), images resized to 1024×1024, with cores strictly pinned using taskset (no OS interference).

• 10 test images per configuration
• 1 warmup run discarded per image
• Latency = mean of timed inference runs only

Detailed results: 4 / 8 / 12 Cores

✅ Zero quality degradation. All ONNX variants (FP32, FP16, Dynamic INT8) achieve IoU = 1.000 and MAE = 0.0004 compared to ground-truth masks — identical to the original PyTorch model.

---

📈 CPU Core Scaling — Dynamic INT8 (1 to 14 Cores)

Observations:

• Scaling is near-linear up to ~8 cores, then hits a RAM bandwidth bottleneck.
• After 12 cores, latency plateaus — additional cores provide no benefit for this model at 1024×1024.
• FP16 is slower than FP32 on CPU (x86 CPUs lack native FP16 compute units).
• Dynamic INT8 leverages AVX2/AVX-512 integer instructions for maximum throughput.

---

🔑 Key Findings

---

🔁 Reproduce the Benchmark

git clone https://huggingface.co/mmahdi-sz/FeyNobg-ONNX
cd FeyNobg-ONNX

# Create virtualenv
python -m venv venv && source venv/bin/activate
pip install onnxruntime numpy pillow

# Run scaling benchmark (1–14 cores, generates CSV + chart)
python hf_scaling_benchmark.py

# Run full 4/8/12 core benchmark (all variants)
python run_benchmark.py

All benchmarks use taskset -c 0-<N-1> to strictly pin CPU cores at the OS level, preventing thread migration and ensuring reproducible measurements.

---

Hardware Tested

---

Citation

If you use this benchmark in your work, please cite the original FeyNoBg model:

@model{feynobg,
  author = {nobg},
  title  = {FeyNoBg: BiRefNet-based Background Removal},
  year   = {2024},
  url    = {https://huggingface.co/nobg/FeyNobg}
}

---

Benchmarks by @mmahdi-sz