tamizhimet-muril-base
google/muril-base-cased fine-tuned to classify a Tamil song-lyric line into one of 9 rhetorical-figure classes (TamizhiMet).
Labels: Literal, Uvamai, Uruvagam, Uyarvu_Navirchi_Ani, Thatkurippetra_Ani, Vanchapugalchi_Ani, Madakku, Ilivu, Iyam.
Training
- Data: TamizhiMet train split, 8,680 lines (one label per line).
- Epochs 5, batch size 32, learning rate 3e-05, AdamW (weight decay 0.01), linear schedule with 10% warmup, max length 64 tokens.
- Cross-entropy with class weights proportional to n_c^-0.5 (mean 1).
- Seed 42; precision fp16; checkpoint after the last epoch.
Results of this checkpoint
Macro-F1 is averaged over the classes that have gold lines in the split.
| split | lines | macro-F1 | accuracy | MCC | classes without gold lines |
|---|---|---|---|---|---|
| val | 1,240 | 0.268 | 0.873 | 0.784 | Madakku, Iyam |
| test | 2,480 | 0.234 | 0.874 | 0.782 | Ilivu |
Per-class F1 (gold support in brackets; fewer than 10 gold lines is unreliable):
| class | val | test |
|---|---|---|
| Literal | 0.89 (668) | 0.90 (1337) |
| Uvamai | 0.98 (429) | 0.98 (871) |
| Uruvagam | 0.00 (105) | 0.00 (194) |
| Uyarvu_Navirchi_Ani | 0.00 (25) | 0.00 (44) |
| Thatkurippetra_Ani | 0.00 (7) | 0.00 (25) |
| Vanchapugalchi_Ani | 0.00 (5) | 0.00 (5) |
| Madakku | n/a (0) | 0.00 (1) |
| Ilivu | 0.00 (1) | n/a (0) |
| Iyam | n/a (0) | 0.00 (3) |
For reference, the paper runs (same settings) score macro-F1 val 0.268 ± 0.000; test 0.234 ± 0.000 (mean ± sd over seeds 42/43/44). This checkpoint is a retrained seed, so its scores differ slightly.
Limitations
The training data is dominated by Literal and Uvamai; the rarest classes have very few training lines, and the model seldom or never predicts them (see the per-class table).
Usage
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer
repo = "kavienanj/tamizhimet-muril-base"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()
line = "பறவை போல் பறந்தவன்"
enc = tok(line.strip() + '', add_special_tokens=True, truncation=True, max_length=64, return_tensors="pt")
with torch.no_grad():
probs = model(**enc).logits.softmax(-1)[0]
print(model.config.id2label[int(probs.argmax())])