ModernBERT-base-biomedical-ner

Model Details

• Model Name: ModernBERT-base-biomedical-ner
• Model Architecture: ModernBERT (Bidirectional Encoder Representations from Transformers)
• Pre-trained Model: answerdotai/ModernBERT-base
• Fine-tuned on: SourceData Dataset
• Fine-tuned by: Kushtrim Visoka

Model Description

The ModernBERT-base-biomedical-ner is a fine-tuned variant of the ModernBERT (Bidirectional Encoder Representations from Transformers) model, designed specifically for the task of Named Entity Recognition (NER) in the biomedical domain. The model has been fine-tuned on the SourceData Dataset, which is a substantial and comprehensive biomedical corpus for machine learning and AI in the publishing context.

Named Entity Recognition is a crucial task in natural language processing, particularly in the biomedical field, where identifying and classifying entities like genes, proteins, diseases, and more is essential for various applications, including information retrieval, knowledge extraction, and data mining.

Intended Use

The ModernBERT-base-biomedical-ner model is intended for NER tasks within the biomedical domain. It can be used for a range of applications, including but not limited to:

• Identifying and extracting biomedical entities (e.g., genes, proteins, diseases) from unstructured text.
• Enhancing information retrieval systems for scientific literature.
• Supporting knowledge extraction and data mining from biomedical literature.
• Facilitating the creation of structured biomedical databases.

Labels

Source of label information: EMBO/SourceData Dataset

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
import pandas as pd

tokenizer = AutoTokenizer.from_pretrained("Kushtrim/ModernBERT-base-biomedical-ner")
model = AutoModelForTokenClassification.from_pretrained("Kushtrim/ModernBERT-base-biomedical-ner")

ner = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy='first')

text = """In a recent study, researchers investigated the effect of aspirin on gene expression in tumor necrosis factor alpha signaling pathways. The compound was observed to localize within the mitochondrial matrix of T-helper cells, which are crucial for adaptive immunity. Tissue samples from the pulmonary epithelium of Mus musculus were analyzed using RNA sequencing to quantify transcriptomic changes. The results showed a notable decrease in markers associated with rheumatoid arthritis progression. These effects were validated in the HeLa cells, confirming the role of aspirin in modulating inflammatory gene networks."""

results = ner(text)
pd.DataFrame.from_records(results)

Development and attribution

• Original ModernBERT architecture and base weights: Answer.AI, through answerdotai/ModernBERT-base
• Biomedical NER fine-tuning, evaluation, packaging, and release: Kushtrim Visoka
• Model card and ongoing maintenance: Kushtrim Visoka

Acknowledgements

This work builds on answerdotai/ModernBERT-base and the work of its original authors. The upstream architecture and base weights remain the work of their respective creators. This repository documents the task-specific adaptation and release work carried out by Kushtrim Visoka and does not claim authorship of the original model. Dataset creators and tool authors referenced elsewhere in this card should also be cited according to their licenses and documentation.

Citation

If you use this fine-tuned model, please cite this repository and the original upstream model.

@misc{visoka_modernbert_base_biomedical_ner_2026,
  author       = {Visoka, Kushtrim},
  title        = {ModernBERT Base Biomedical NER},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/Kushtrim/ModernBERT-base-biomedical-ner},
  note         = {Fine-tuned derivative of answerdotai/ModernBERT-base}
}