gliner-bi-small-v2.0, ONNX
ONNX export of knowledgator/gliner-bi-small-v2.0, a GLiNER bi-encoder (108M parameters: ettin-encoder-68m for the text, all-MiniLM-L12-v2 for the entity types). It finds entities of any type you name, in one forward pass, and runs in ONNX Runtime on the CPU and in the browser (WebAssembly).
| File | Size | |
|---|---|---|
onnx/model_q8.onnx |
117 MB | 8-bit: MatMulNBits weights (block 32, symmetric) and 8-bit embedding tables |
onnx/model.onnx |
429 MB | fp32 reference |
Both are one graph with the label encoder inside: GLiNER's export_to_onnx (bi-encoder, labels as token ids).
Inputs
| Name | Shape | |
|---|---|---|
input_ids, attention_mask |
[1, tokens] |
the text, split into words with \w+(?:[-_]\w+)*|\S and each word tokenized on its own (is_split_into_words), with the tokenizer's special tokens around it |
words_mask |
[1, tokens] |
1-based word index on each word's first sub-token, 0 elsewhere |
text_lengths |
[1, 1] |
number of words |
span_idx, span_mask |
[1, words × 12, 2], [1, words × 12] |
every span of 1 to 12 words; the mask drops those past the last word |
labels_input_ids, labels_attention_mask |
[types, label tokens] |
the entity types, tokenized with all-MiniLM-L12-v2's tokenizer, padded |
logits is [1, words, 12, types]. Take the sigmoid, keep spans at or above a threshold, then, best first, every span
that does not overlap one already kept. reference.json holds four texts with their exact inputs and the entities the
fp32 graph returns.
Checks
On 40 labelled workplace texts (tickets, roadmap notes, postmortems, store and warehouse handovers, customer messages) in English, Swedish, German, Polish and Dutch, with the types person, team or department, product, software system or tool, date, location and ID number, at threshold 0.3:
| Entity F1 | English | Entities differing from PyTorch | |
|---|---|---|---|
| PyTorch | 0.813 | ||
model.onnx |
0.813 | 0 | |
model_q8.onnx |
0.814 | 0.856 | 12 |
Plain dynamic uint8 quantization of the whole graph is about as small, but changed many more entities and ran slower on the CPU.
Apache-2.0, as the original model. Model by Knowledgator.