Developed at Agile Lab by Luca Sorrentino and Annalisa Belia, with Irene Donato as project lead.
Model Name: gliner-contractner-multi-v2.1 (Agile Lab Fine-tune)
Base Architecture: GLiNER Multi v2.1 (Backbone: microsoft/mdeberta-v3-base)
Dataset: agilelab-org/ContractNER
GLiNER ContractNER Multi is a multilingual span-based Named Entity Recognition (NER) model fine-tuned by Agile Lab on the ContractNER dataset. It is designed to extract fine-grained entities from legal contracts with high precision.
Built on the GLiNER Multi v2.1 architecture, this model achieves 72.49% micro F1 on the held-out test set, significantly outperforming general-purpose LLMs and domain-specific legal models in our benchmarks.
You can try it on our HF Space: https://huggingface.co/spaces/agilelab-org/ContractNerSpace
To use this model, you need to install the gliner library.
pip install gliner
from gliner import GLiNER
model = GLiNER.from_pretrained("agilelab-org/Contractner")
# Example contract text
text = """
This EMPLOYMENT AGREEMENT is made effective as of January 1, 2026,
by and between Tech Solutions Inc. ("Company") and Jane Doe ("Executive").
The Executive shall serve as Chief Technology Officer.
The Company agrees to pay the Executive an annual base salary of $250,000.00.
"""
# Define the entities you want to extract (Open Vocabulary)
labels = [
"Parties", "EffectiveDate", "Role", "Salary", "TerminationDate"
]
# Predict
entities = model.predict_entities(text, labels, threshold=0.5)
# Print results
for entity in entities:
print(f"{entity['text']} => {entity['label']} (Score: {entity['score']:.2f})")
Evaluated on the held-out test set (158 samples, 20% stratified split, not seen during training).
Reproduce with: uv run eval.py from the model repo.
Methodology: overlap_cover matching — a prediction counts as TP if it covers 100% of the gold span (boundary tolerance ±1 char), greedy 1-to-1 matching, micro-averaged P/R/F1.
| Metric | Score |
|---|---|
| Micro F1 | 72.49% |
| Micro Precision | 83.13% |
| Micro Recall | 64.27% |
| Macro F1 | 72.94% |
| Approach | F1 |
|---|---|
| GLiNER ContractNER (this model) | 72.49% |
| Standalone DeBERTa models | 46–78% |
| General-purpose LLMs (Qwen, Gemma) | < 35% |
| Legal-specific models (LegalBERT, ContractBERT) | < 10% |
| Entity | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| PII_Ref | 94.12% | 94.12% | 94.12% | 17 |
| EffectiveDate | 95.83% | 88.46% | 92.00% | 26 |
| Salary | 89.47% | 94.44% | 91.89% | 18 |
| Percentage | 91.30% | 87.50% | 89.36% | 24 |
| Role | 96.67% | 82.86% | 89.23% | 35 |
| Parties | 85.71% | 87.10% | 86.40% | 62 |
| Shares | 100.00% | 73.68% | 84.85% | 19 |
| Price | 92.31% | 75.00% | 82.76% | 16 |
| Court | 83.33% | 75.00% | 78.95% | 20 |
| RenewalTerm | 66.67% | 83.33% | 74.07% | 12 |
| Principal | 82.61% | 61.29% | 70.37% | 31 |
| Title | 82.46% | 60.26% | 69.63% | 78 |
| Address | 62.96% | 77.27% | 69.39% | 22 |
| TerminationDate | 71.43% | 55.56% | 62.50% | 18 |
| Rent | 66.67% | 50.00% | 57.14% | 8 |
| Ratio | 85.71% | 35.29% | 50.00% | 17 |
| Act | 85.00% | 25.00% | 38.64% | 68 |
| Regulation | 47.37% | 23.68% | 31.58% | 38 |
Note on Act and Regulation: low recall is due to truncation — these entities often appear in long legal clauses that exceed the model's 384-token context window. Use sliding window inference for full-document coverage.
The model was trained on the ContractNER schema. While you can use custom labels, performance is best with categories semantically similar to:
RevolvingCredit class (too rare and ambiguous).microsoft/mdeberta-v3-base (86M backbone + 190M embedding parameters).When a law firm or investment fund needs to analyze large volumes of contracts ahead of an acquisition or investment, this model accelerates the extraction of key information—parties involved, relevant dates, renewal terms, and financial values—reducing manual pre-screening effort and narrowing the scope of documents that require deep legal review.
Relevant entities: Parties, EffectiveDate, TerminationDate, RenewalTerm, Salary, Price, Principal
By extracting EffectiveDate, TerminationDate, and RenewalTerm across a contract portfolio, organizations can power alert systems that proactively surface upcoming expirations, auto-renewals, and notice windows—improving operational control over contractual obligations at scale.
Relevant entities: EffectiveDate, TerminationDate, RenewalTerm
When a legal team needs to assess the impact of new regulatory requirements or internal policy changes across active contracts, the model helps quickly identify documents containing regulatory references, specific parties, and relevant dates—narrowing the audit perimeter to contracts that actually warrant deeper review.
Relevant entities: Act, Regulation, Court, Parties, EffectiveDate
model.onnx) is included in the repository.Hardware: MacBook Pro — Apple M1 Pro (8-core, 6P+2E), 16 GB RAM
Runs: 20 (after 3 warmup) · Threshold: 0.9
| Backend | Input length | Median | p95 | Throughput |
|---|---|---|---|---|
| PyTorch CPU | Short (~300 chars) | 134 ms | 148 ms | ~7.4 docs/s |
| PyTorch CPU | Medium (~800 chars) | 202 ms | 216 ms | ~5.0 docs/s |
| PyTorch CPU | Full chunk (~1,000 chars) | 261 ms | 275 ms | ~3.8 docs/s |
| ONNX Runtime CPU | Full chunk (~1,000 chars) | 308 ms | — | ~3.2 docs/s |
ONNX Runtime on Apple M1 is slightly slower than PyTorch due to Apple Silicon optimizations in PyTorch's ARM kernels. On x86 Linux (e.g., AWS c5), ONNX typically delivers 1.5–2× speedup over PyTorch CPU.
For documents longer than 384 tokens, use sliding window inference to avoid truncation.
| Approach | F1 on ContractNER | Cost / 1,000 docs |
|---|---|---|
| This model (CPU, self-hosted) | 72.49% | ≈$0.014 |
| GPT-4o-mini | < 35% | ≈$0.07 |
| GPT-4o | < 35% | ≈$0.33 |
Self-hosted cost assumes AWS c5.xlarge on-demand ($0.17/hr, ≈3.8 docs/s, full utilization). LLM costs estimated at list pricing (≈250 input + ≈50 output tokens per chunk).
Apache 2.0
model.onnx 1.2 GBpytorch_model.bin 1.2 GB