⚽ SOCCER BERT
A multilingual football-specific BERT model trained from scratch.
Model Details
| Property | Value |
|---|---|
| Parameters | 159M |
| Architecture | Decoder-only Transformer |
| Layers | 10 |
| Attention Heads | 8 |
| Embedding Dim | 512 |
| Context Length | 512 tokens |
| Vocab Size | 250,002 (XLM-RoBERTa) |
| Training Phase | 2 of 3 |
| Dataset | 3.72 GB football + wikipedia |
| Format | Safetensors |
| Status | Private (Phase 3 SFT pending) |
Languages
English, Spanish, Portuguese, Arabic, Bengali, Hindi
Citation
@misc{Soccer-BERT-2026,
author = {Tareq},
title = {Soccer BERT},
year = {2025},
url = {https://huggingface.co/tareq052/soccer-bert}
}