BERTimbau Legal TJSC

Fine-tuned BERT model for Brazilian legal text classification across 5 areas of law. Part of the LegalBench-BR project — the first public benchmark for LLM evaluation on Brazilian legal text classification.

📄 Paper: Zenodo DOI 10.5281/zenodo.19298367 📊 Dataset: pedronettotrue/legal-nlp-benchmark-br

---

Model Description

• Base model: neuralmind/bert-base-portuguese-cased (BERTimbau)
• Fine-tuning method: LoRA (merged into base — no PEFT required)
• Task: Multi-class legal text classification
• Language: Portuguese (Brazilian)
• Accuracy: 87.6% on TJSC test set (+28pp over GPT-4o mini)

---

Labels

---

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "pedronettotrue/bertimbau-legal-tjsc"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

texto = "Classe: Apelação Criminal. Assuntos: Tráfico de Drogas. Órgão: 4ª Câmara Criminal."
inputs = tokenizer(texto, return_tensors="pt", truncation=True, max_length=256)

with torch.no_grad():
    pred = model(**inputs).logits.argmax(dim=1).item()

print(model.config.id2label[pred])  # → "penal"

---

Training Details

• Dataset: TJSC (Tribunal de Justiça de Santa Catarina) court decisions
• Fine-tuning: LoRA (r=16, alpha=32) on classification head
• Merged weights: yes — load directly with AutoModelForSequenceClassification, no PEFT needed
• Hardware: GPU cloud (Vast.ai)

---

Citation

@misc{neto2026legalbenchbr,
  title={LegalBench-BR: Benchmark for LLM Evaluation on Brazilian Legal Text Classification},
  author={Pedro Neto},
  year={2026},
  doi={10.5281/zenodo.19298367},
  url={https://doi.org/10.5281/zenodo.19298367}
}

---

Author

Pedro Neto — AI Engineer | Atlas AI Nexus LinkedIn · HuggingFace