kk-bert-small-bpe-kaznerd
BERT-Small finetuned on KazNERD (Kazakh Named Entity Recognition) starting from
Abzalbek89/kk-bert-small-bpe — pretrained with the bpe tokenizer.
Part of a comparative study of subword tokenizers for Kazakh small language models.
Test set results
| metric | value |
|---|---|
| F1 (span-level, seqeval) | 86.96 |
| Precision | 85.22 |
| Recall | 88.77 |
| Token accuracy | 97.55 |
Training
| Base model | Abzalbek89/kk-bert-small-bpe |
| Dataset | IS2AI/KazNERD (IOB2, 25 entity types) |
| Epochs | 5 |
| Batch size | 16 |
| Learning rate | 5e-5 (warmup 10%, weight decay 0.01) |
| Precision | fp16 |
| Hardware | RTX A4000 |
| Time | 9.0 min |
| Seed | 42 |
Usage
from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline
tok = AutoTokenizer.from_pretrained("Abzalbek89/kk-bert-small-bpe-kaznerd")
model = AutoModelForTokenClassification.from_pretrained("Abzalbek89/kk-bert-small-bpe-kaznerd")
ner = pipeline("ner", model=model, tokenizer=tok, aggregation_strategy="simple")
ner("Қазақстан Республикасының Президенті Қасым-Жомарт Тоқаев Алматыға барды.")
Companion artifacts
- Pretrained base:
Abzalbek89/kk-bert-small-bpe - Aggregate metrics:
Abzalbek89/kk-tokenizer-fertility-baseline