ModernBERT-large for primary legal-topic classification
Paper: Legal Guidance in the Wild | Code: GitHub
This checkpoint assigns a primary legal topic to conversations in which the user is seeking legal guidance. It is the second stage of the full-conversation ModernBERT-large cascade from the Legal Guidance in the Wild study. Input contains chronological user messages only; assistant messages are excluded. The first-stage detector is modernbert-large-v3-seeks-guidance-user-len4096-seed42.
This is a research classifier, not a legal-advice system. Its topic prediction must not be treated as a determination of jurisdiction, rights, or legal merit.
Labels
The 14 labels are: FAMILY_AND_ESTATES, HOUSING_AND_PROPERTY,
EMPLOYMENT_AND_LABOR, IMMIGRATION_AND_CITIZENSHIP, CRIMINAL_LAW,
TAX_LAW, CONSUMER_AND_PERSONAL_FINANCE, BUSINESS_AND_COMMERCIAL,
TORTS_AND_CIVIL_LIABILITY, GOVERNMENT_AND_ADMINISTRATIVE,
CIVIL_RIGHTS_AND_CONSTITUTIONAL, DATA_PRIVACY_AND_TECHNOLOGY,
INTELLECTUAL_PROPERTY, and OTHER. The exact ID mapping is stored in
config.json.
Data
- Dataset: AmirMohseni/WildChat-Legal-Classification-V3-Hierarchical
- Requested revision:
main(latest at run time) - Train fingerprint:
6ba4c2696e889276 - Validation fingerprint:
403fe118d76c8360 - Topic-stage train / validation rows: 750 / 134 guidance-positive conversations
- Input mode: chronological user messages only
Dataset access follows the linked repository's sharing settings. The
fingerprints identify the exact loaded splits even if main later changes.
Training configuration
| Setting | Value |
|---|---|
| Base model | answerdotai/ModernBERT-large |
| Maximum length | 4096 |
| Epochs | 10 |
| Learning rate | 6e-05 |
| Effective batch size | 32 |
| Weight decay | 0.01 |
| Class weighting | Yes |
| Seed | 42 |
| Hardware | NVIDIA A100-SXM4-40GB |
The checkpoint was selected by silver-validation macro-F1.
Silver-validation results
| Evaluation | Accuracy | Macro-F1 |
|---|---|---|
| Conditional topic stage (N=134) | 0.716 | 0.574 |
| Full large cascade (15-way, N=290) | 0.776 | 0.536 |
The paired first-stage guidance detector obtains 0.893 macro-F1 and 0.887 positive-class F1. Gold-set results are reported below.
Gold-set results
Scored once on the 200 human-adjudicated gold conversations after all modeling decisions were frozen (paper, Tables 4 and 6; per-example predictions are in the GitHub repository).
| Evaluation | Accuracy | Macro-F1 |
|---|---|---|
| Guidance (N=200) | 0.845 | 0.845 |
| Topic, gold guidance-positive (N=84) | 0.762 | 0.671 |
| Full large cascade (N=200) | 0.755 | 0.593 |
Inference
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "AmirMohseni/modernbert-large-v3-primary-topic-user-len4096-seed42"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo)
# Run only after the paired guidance detector predicts guidance-seeking.
user_only_conversation = "\n".join(
f"User: {m['content']}" for m in conversation if m["role"] == "user"
)
inputs = tokenizer(user_only_conversation, return_tensors="pt", truncation=True,
max_length=4096)
topic_id = int(model(**inputs).logits.argmax(-1)[0])
topic = model.config.id2label[topic_id]
Limitations
The model was trained on English-language public LLM interaction logs with silver labels, one source, and one seed. Several topics have limited validation support, so macro-F1 is unstable and topic-specific errors can be substantial. Inputs beyond 4,096 tokens are truncated. The taxonomy is jurisdiction-agnostic and cannot substitute for legal triage by a qualified professional. Human review is required for consequential use.
Citation
@inproceedings{mohseni2026legal,
title = {Legal Guidance in the Wild: How Users Seek Legal Help in Real-World {LLM} Conversations},
author = {Mohseni, Amirreza and Spanakis, Gerasimos},
booktitle = {Legal Knowledge and Information Systems (JURIX 2026)},
year = {2026},
eprint = {2610.25159},
archivePrefix = {arXiv}
}