Model card for OpenBioNER-base-v2

OpenBioNER-base-v2 is the flagship model of the OpenBioNER-v2 family, offering a compact, description-driven encoder for zero-shot biomedical named entity recognition (NER). Like its predecessor openbioner-base, it identifies unseen entity types using only their natural language descriptions, removing the need for retraining or label-specific fine-tuning.

Available Models

Model Details

OpenBioNER-base-v2 is trained with a new two-stage pipeline that combines large-scale ontology-based pretraining with a targeted refinement phase on high-quality biomedical annotations.

In the first stage -BroadScan-, we employ LLaMA-3.1-8B-instruct to generate multi-view descriptions foreach entity type. Prompts are designed to elicit short summaries that capture the core semantics of the type across various biomedical subfields and contexts.

In the second stage -BioRefine-, we shift to single-view descriptions, prompting the model to generate short, high-level summaries that focus on the typical scenario in which the entity is expected to appear.

This design delivers state-of-the-art zero-shot performance at both the token and entity level across 10 biomedical benchmarks, including challenging datasets such as BioRED and MedMentions-R, under a stricter zero-shot protocol that removes in-domain fine-tuning data.

Extensive comparisons against 19 state-of-the-art systems—from compact encoders like GLiNER to large-scale models like UniversalNER—show that OpenBioNER-base-v2 consistently outperforms much larger LLMs and vast majority of GliNER baselines rivaling at entity level only with GliNER-Biomed-bi-large, a 4,5x bigger model specifically designed for the domain, while showing more robust perfomance on rarest type and remaining lightweight, open, and production-ready at just 110M parameters.

Installation

To use this model, you must install the IBM Zshot library:

!pip install -U zshot==0.0.11 datasets gliner
!python -m spacy download en_core_web_sm

Usage

import spacy

from zshot import PipelineConfig, displacy
from zshot.linker import LinkerSMXM
from zshot.evaluation.metrics._seqeval._seqeval import Seqeval
from zshot.utils.data_models import Entity
from zshot.evaluation.zshot_evaluate import evaluate, prettify_evaluate_report

# define your list of candidate entity types
entities = [
     Entity(name='BACTERIUM', description='A bacterium refers to a type of microorganism that can exist as a single cell and may cause infections or play a role in various biological processes. Examples include species like Streptococcus pneumoniae and Streptomyces ahygroscopicus.', vocabulary=None),
]

nlp = spacy.blank("en")
nlp_config = PipelineConfig(
    linker=LinkerSMXM(model_name="disi-unibo-nlp/openbioner-base-v2"),
    entities=entities,
    device='cuda' # or 'cpu' if GPU not available
)
nlp.add_pipe("zshot", config=nlp_config, last=True)


sentence = "Impact of cofactor - binding loop mutations on thermotolerance and activity of E. coli transketolase"
doc = nlp(sentence)

displacy.render(doc, style="ent")

Run Evaluation

Let's evaluate the model on NCBI dataset

from datasets import load_dataset
ds = load_dataset('disi-unibo-nlp/ncbi', split='test')
print("Tokens:", ds['tokens'][0])
print("Tags:", ds['ner_tags'][0])
print("Unique labels:", set([t[2:] for tag in ds['ner_tags'] for t in tag if t != 'O']))

Tokens: ['Clustering', 'of', 'missense', 'mutations', 'in', 'the', 'ataxia', '-', 'telangiectasia', 'gene', 'in', 'a', 'sporadic', 'T', '-', 'cell', 'leukaemia', '.']
Tags: ['O', 'O', 'O', 'O', 'O', 'O', 'B-DISEASE', 'I-DISEASE', 'I-DISEASE', 'O', 'O', 'O', 'B-DISEASE', 'I-DISEASE', 'I-DISEASE', 'I-DISEASE', 'I-DISEASE', 'O']
Unique labels: {'DISEASE'}

Evaluation with zshot library

import spacy

from zshot import PipelineConfig, displacy
from zshot.linker import LinkerSMXM
from zshot.evaluation.metrics._seqeval._seqeval import Seqeval
from zshot.utils.data_models import Entity
from zshot.evaluation.zshot_evaluate import evaluate, prettify_evaluate_report



entities = [
  Entity(name='DISEASE', description='A disease is a medical condition that disrupts normal bodily functions or structures, affecting various organs or systems, and leading to symptoms like muscle weakness, fatigue, stiffness, or cognitive impairment. Diseases can impact muscles, the nervous system, heart, eyes, and more, and may be chronic or acute, such as diabetes, cardiovascular or neurological disorders, and cancer-related conditions like lymphoblastic leukemia or lymphoma.', vocabulary=None),
]

nlp = spacy.blank("en")
nlp_config = PipelineConfig(
    linker=LinkerSMXM(model_name="disi-unibo-nlp/openbioner-base-v2"),
    entities=entities,
    device='cuda'
)

nlp.add_pipe("zshot", config=nlp_config, last=True)
print("Evaluating...")
evaluation = evaluate(nlp, ds, metric=Seqeval())
print("Done!")

print(prettify_evaluate_report(evaluation)[0])

Evaluating...
Done!
+-----------------------------------+
|             linker -              |
|       General - span-based        |
+-------------------------+---------+
|          Metric         |         |
+-------------------------+---------+
| overall_precision_micro |  0.6584 |
|   overall_recall_micro  |  0.6817 |
|     overall_f1_micro    |  0.6681 |
| overall_precision_macro |  0.6584 |
|   overall_recall_macro  |  0.6817 |
|     overall_f1_macro    |  0.6681 |
|     overall_accuracy    |  0.9595 |
|  total_time_in_seconds  | 14.9177 |
|    samples_per_second   | 63.0124 |
|    latency_in_seconds   |  0.0159 |
+-------------------------+---------+

Performance

The updated OpenBioNER-base-v2 model achieves state-of-the-art performance, surpassing all the considered baselines across most biomedical NER benchmarks, at both the token and span levels.

Span-level F1 micro scores

Token-level F1 micro scores

⚠️ Note: All results above were computed using the zshot library (v0.0.11), which supports both GLiNER and OpenBioNER architectures. For all GLiNER models, evaluations were performed using lowercase type names and a threshold of 0.5. To reproduce the results, please refer to our GitHub repository.

Descriptions

Descriptions used to evaluate OpenBioNER-base-v2 for each dataset.

---

Negative Class

This is the description used as NEG class (e.g. not an entity) for all the datasets, execept for MedMentions-Rare:

Coal, water, oil, etc. are normally used for traditional electricity generation. However using liquefied natural gas as fuel for joint circulatory electricity generation has advantages. The chief financial officer is the only one there taking the fall. It has a very talented team, eh. What will happen to the wildlife? I just tell them, you've got to change. They're here to stay. They have no insurance on their cars. What else would you like? Whether holding an international cultural event or setting the city's cultural policies, she always asks for the participation or input of other cities and counties.

---

AnatEM

---

BC2GM

---

BC4CHEMD

---

BC5CDR

---

BioRED

---

JNLPBA

---

NCBI

---

MedMentions-Rare (Dev)

---

MedMentions-Rare (Test)

---

JNLPBA-Rare

---

🧬 How to Write Effective Entity Type Descriptions

Entity type descriptions are crucial for improving generalization in OpenBioNER. Well-written descriptions help models disambiguate types, handle rare classes, and align with real-world usage across diverse datasets.

✅ Best Practices

• Start with a clear definition: Briefly explain what the entity type is.

• Include functions or context: Add what it does, its purpose, or where it appears.

• List 3–5 concrete examples: Use domain-relevant examples (e.g., real diseases, proteins, or food items).

• Mention subtypes or synonyms (optional): Helps capture lexical variation and rare mentions.

• Keep it concise: 1–3 well-structured sentences are ideal.

⚠️ Common Mistakes to Avoid

• Vague or overly generic descriptions
• No examples
• Just a list of terms
• Redundant or circular wording

🧪 Template (Recommended Format)

A [TYPE] refers to [concise definition]. It includes examples such as [example1], [example2], and [example3].

Authors

• Alessio Cocchieri
• Giacomo Frisoni
• Francesco Zangrillo
• Luca Ragazzi
• Marcos Martinez Galindo
• Gianluca Moro
• Giuseppe Tagliavini

📬 Contacts

For questions, collaborations, or feedback, feel free to reach out:

• Alessio: a.cocchieri@unibo.it
• Giacomo: giacomo.frisoni@unibo.it
• Francesco: f.zangrillo@unibo.it