DeBERTa-v3-base-mnli-fever-docnli-ling-2c-onnx

fp32 ONNX export of MoritzLaurer/DeBERTa-v3-base-mnli-fever-docnli-ling-2c, a DeBERTa-v3-base NLI cross-encoder, packaged for CPU inference with ONNX Runtime.

This repository only re-packages the base model as ONNX — no additional training was performed. The export is numerically faithful to the original (entailment probabilities match PyTorch to 3 decimals).

Files

Self-contained (ONNX graph + tokenizer + config).

Why no INT8 build? Dynamic INT8 quantization severely degrades this model — DeBERTa's disentangled-attention ops are quantization-sensitive, and entailment probabilities collapse (e.g. a near-verbatim match drops from ~0.99 to ~0.49). Serve fp32. On CPU this ~184M model runs in tens of milliseconds per pair, so quantization buys little anyway.

Labels

Binary NLI (2-class). Apply softmax to the logits; the entailment probability is softmax(logits)[0].

Usage

import onnxruntime as ort, numpy as np
from transformers import AutoTokenizer

repo = "Hanno-Labs/DeBERTa-v3-base-mnli-fever-docnli-ling-2c-onnx"
tok = AutoTokenizer.from_pretrained(repo, subfolder="nli-onnx")
sess = ort.InferenceSession("nli-onnx/model.onnx", providers=["CPUExecutionProvider"])

enc = tok("A cat sat on the mat.", "There is a cat.", return_tensors="np", truncation=True)
logits = sess.run(None, {k: v for k, v in enc.items()})[0]
probs = np.exp(logits) / np.exp(logits).sum(-1, keepdims=True)
print("entailment prob:", float(probs[0, 0]))

(Also loadable via optimum.onnxruntime.ORTModelForSequenceClassification.from_pretrained(repo, subfolder="nli-onnx").)

How this was exported

optimum-cli export onnx \
  --model MoritzLaurer/DeBERTa-v3-base-mnli-fever-docnli-ling-2c \
  --task text-classification --optimize O3 nli-onnx/

License & attribution

MIT, inherited from the base model. Base architecture: DeBERTa-v3-base (Microsoft). Packaged by Hanno-Labs.

If you use this model, please cite the original author:

Laurer, Moritz, Wouter van Atteveldt, Andreu Salleras Casas, and Kasper Welbers. 2022. Less Annotating, More Classifying – Addressing the Data Scarcity Issue of Supervised Machine Learning with Deep Transfer Learning and BERT - NLI.