Model card for OpenBioNER-compact-v2

OpenBioNER-compact-v2 is the lightweight version of the OpenBioNER-v2 family, offering a 65M parameter encoder for biomedical NER while retaining the description-driven zero-shot design of its larger sibling.

Available Models

Model Details

This model provides a strong trade-off between accuracy and efficiency, maintaining competitive performance across biomedical NER benchmarks while being 2× smaller than OpenBioNER-base-v2, making it suitable for:

• Resource-constrained deployments (e.g., edge devices, low-memory GPUs, production pipelines).

• High-throughput annotation of large biomedical corpora.

• Real-time applications where speed is critical.

Although compact, the model still leverages the two-stage training pipeline of openbioner-base-v2, including large-scale ontology-based pretraining and refinement on high-quality biomedical annotations. It achieves strong zero-shot performance and remains fully open, MIT-licensed, and production-ready.

Links

Installation

To use this model, you must install the IBM Zshot library:

!pip install -U zshot==0.0.11 datasets gliner
!python -m spacy download en_core_web_sm

Usage

import spacy

from zshot import PipelineConfig, displacy
from zshot.linker import LinkerSMXM
from zshot.evaluation.metrics._seqeval._seqeval import Seqeval
from zshot.utils.data_models import Entity
from zshot.evaluation.zshot_evaluate import evaluate, prettify_evaluate_report

# define your list of candidate entity types
entities = [
     Entity(name='BACTERIUM', description='A bacterium refers to a type of microorganism that can exist as a single cell and may cause infections or play a role in various biological processes. Examples include species like Streptococcus pneumoniae and Streptomyces ahygroscopicus.', vocabulary=None),
]

nlp = spacy.blank("en")
nlp_config = PipelineConfig(
    linker=LinkerSMXM(model_name="disi-unibo-nlp/openbioner-compact-v2"),
    entities=entities,
    device='cuda' # or 'cpu' if GPU not available
)
nlp.add_pipe("zshot", config=nlp_config, last=True)


sentence = "Impact of cofactor - binding loop mutations on thermotolerance and activity of E. coli transketolase"
doc = nlp(sentence)

displacy.render(doc, style="ent")

Run Evaluation

Let's evaluate the model on NCBI dataset

from datasets import load_dataset
ds = load_dataset('disi-unibo-nlp/ncbi', split='test')
print("Tokens:", ds['tokens'][0])
print("Tags:", ds['ner_tags'][0])
print("Unique labels:", set([t[2:] for tag in ds['ner_tags'] for t in tag if t != 'O']))

Tokens: ['Clustering', 'of', 'missense', 'mutations', 'in', 'the', 'ataxia', '-', 'telangiectasia', 'gene', 'in', 'a', 'sporadic', 'T', '-', 'cell', 'leukaemia', '.']
Tags: ['O', 'O', 'O', 'O', 'O', 'O', 'B-DISEASE', 'I-DISEASE', 'I-DISEASE', 'O', 'O', 'O', 'B-DISEASE', 'I-DISEASE', 'I-DISEASE', 'I-DISEASE', 'I-DISEASE', 'O']
Unique labels: {'DISEASE'}

Evaluation with zshot library

import spacy

from zshot import PipelineConfig, displacy
from zshot.linker import LinkerSMXM
from zshot.evaluation.metrics._seqeval._seqeval import Seqeval
from zshot.utils.data_models import Entity
from zshot.evaluation.zshot_evaluate import evaluate, prettify_evaluate_report



entities = [
  Entity(name='DISEASE', description='A disease is a medical condition that disrupts normal bodily functions or structures, affecting various organs or systems, and leading to symptoms like muscle weakness, fatigue, stiffness, or cognitive impairment. Diseases can impact muscles, the nervous system, heart, eyes, and more, and may be chronic or acute, such as diabetes, cardiovascular or neurological disorders, and cancer-related conditions like lymphoblastic leukemia or lymphoma.', vocabulary=None),
]

nlp = spacy.blank("en")
nlp_config = PipelineConfig(
    linker=LinkerSMXM(model_name="disi-unibo-nlp/openbioner-compact-v2"),
    entities=entities,
    device='cuda'
)

nlp.add_pipe("zshot", config=nlp_config, last=True)
print("Evaluating...")
evaluation = evaluate(nlp, ds, metric=Seqeval())
print("Done!")

print(prettify_evaluate_report(evaluation)[0])

Evaluating...
Done!
+------------------------------------+
|              linker -              |
|        General - span-based        |
+-------------------------+----------+
|          Metric         |          |
+-------------------------+----------+
| overall_precision_micro |  0.6388  |
|   overall_recall_micro  |  0.5536  |
|     overall_f1_micro    |  0.5931  |
| overall_precision_macro |  0.6388  |
|   overall_recall_macro  |  0.5536  |
|     overall_f1_macro    |  0.5931  |
|     overall_accuracy    |  0.9573  |
|  total_time_in_seconds  |  8.5039  |
|    samples_per_second   | 110.5372 |
|    latency_in_seconds   |  0.0090  |
+-------------------------+----------+

Performance

The compact model delivers state-of-the-art performance for its size, ranking second best at token level and generally above many larger models and rivaling medium-size encoders like GLiNER-bi-base while being 3× smaller.

Span-level F1 micro scores

Token-level F1 micro scores

⚠️ Note: All results above were computed using the zshot library (v0.0.11), which supports both GLiNER and OpenBioNER architectures. For all GLiNER models, evaluations were performed using lowercase type names and a threshold of 0.5. To reproduce the results, please refer to our GitHub repository.

Descriptions used to evaluate OpenBioNER-compact-v2 for each dataset.

---

Negative Class

This is the description used as NEG class (e.g. not an entity) for all the datasets, execept for MedMentions-Rare:

Coal, water, oil, etc. are normally used for traditional electricity generation. However using liquefied natural gas as fuel for joint circulatory electricity generation has advantages. The chief financial officer is the only one there taking the fall. It has a very talented team, eh. What will happen to the wildlife? I just tell them, you've got to change. They're here to stay. They have no insurance on their cars. What else would you like? Whether holding an international cultural event or setting the city's cultural policies, she always asks for the participation or input of other cities and counties.

---

AnatEM

---

BC2GM

---

BC4CHEMD

---

BC5CDR

---

BioRED

---

JNLPBA

---

NCBI

---

MedMentions-Rare (Dev)

---

MedMentions-Rare (Test)

---

JNLPBA-Rare

---

🧬 How to Write Effective Entity Type Descriptions

Entity type descriptions are crucial for improving generalization in OpenBioNER. Well-written descriptions help models disambiguate types, handle rare classes, and align with real-world usage across diverse datasets.

✅ Best Practices

• Start with a clear definition: Briefly explain what the entity type is.

• Include functions or context: Add what it does, its purpose, or where it appears.

• List 3–5 concrete examples: Use domain-relevant examples (e.g., real diseases, proteins, or food items).

• Mention subtypes or synonyms (optional): Helps capture lexical variation and rare mentions.

• Keep it concise: 1–3 well-structured sentences are ideal.

⚠️ Common Mistakes to Avoid

• Vague or overly generic descriptions
• No examples
• Just a list of terms
• Redundant or circular wording

🧪 Template (Recommended Format)

A [TYPE] refers to [concise definition]. It includes examples such as [example1], [example2], and [example3].

Authors

• Alessio Cocchieri
• Giacomo Frisoni
• Francesco Zangrillo
• Luca Ragazzi
• Marcos Martinez Galindo
• Gianluca Moro
• Giuseppe Tagliavini

📬 Contacts

For questions, collaborations, or feedback, feel free to reach out:

• Alessio: a.cocchieri@unibo.it
• Giacomo: giacomo.frisoni@unibo.it
• Francesco: f.zangrillo@unibo.it