kk-bert-small-unigram-kaznerd
BERT-Small finetuned on KazNERD (Kazakh Named Entity Recognition) starting from
Abzalbek89/kk-bert-small-unigram — pretrained with the unigram tokenizer.
Part of a comparative study of subword tokenizers for Kazakh small language models.
Test set results
| metric | value |
|---|---|
| F1 (span-level, seqeval) | 87.99 |
| Precision | 86.36 |
| Recall | 89.68 |
| Token accuracy | 97.75 |
Training
| Base model | Abzalbek89/kk-bert-small-unigram |
| Dataset | IS2AI/KazNERD (IOB2, 25 entity types) |
| Epochs | 5 |
| Batch size | 16 |
| Learning rate | 5e-5 (warmup 10%, weight decay 0.01) |
| Precision | fp16 |
| Hardware | RTX A4000 |
| Time | 13.6 min |
| Seed | 42 |
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
tok = AutoTokenizer.from_pretrained("Abzalbek89/kk-bert-small-unigram-kaznerd")
model = AutoModelForTokenClassification.from_pretrained("Abzalbek89/kk-bert-small-unigram-kaznerd")
ner = pipeline("ner", model=model, tokenizer=tok, aggregation_strategy="simple")
ner("Қазақстан Республикасының Президенті Қасым-Жомарт Тоқаев Алматыға барды.")
Companion artifacts
- Pretrained base:
Abzalbek89/kk-bert-small-unigram - Aggregate metrics:
Abzalbek89/kk-tokenizer-fertility-baseline