mDeBERTa-Hybrid-30k
A vocabulary-pruned version of microsoft/mdeberta-v3-base with a 30k-token hybrid Indonesian–English vocabulary, designed for downstream Indonesian and Indonesian–English NLP tasks.
This model was developed using VocabPrune, a deterministic, language-aware, frequency-based vocabulary pruning method designed to reduce vocabulary-related model overhead while preserving the original Transformer architecture.
The hybrid vocabulary uses an approximately 70% Indonesian / 30% English configuration.
The model is a base checkpoint and should be fine-tuned for a specific downstream task.
Model Details
| Property | Value |
|---|---|
| Base model | microsoft/mdeberta-v3-base |
| Vocabulary size | 30k tokens |
| Vocabulary | Hybrid Indonesian–English |
| Language focus | Indonesian with English subword coverage |
| Architecture | mDeBERTa-v3-base |
Resources
- Paper: IEEE Access
- Paper DOI: 10.1109/ACCESS.2026.3679735
- Source Code: GitHub – VocabPrune
- Reproducibility Archive: Zenodo – VocabPrune v1.0.1
- Model Collection: Pruned mDeBERTa
For the methodology, experimental setup, and detailed evaluation results, please refer to the published paper.
Citation
If you use this model or the VocabPrune methodology in your research, please cite:
@article{fuadi2026efficient,
author = {Fuadi, Mukhlish and Wibawa, Adhi Dharma and Sumpeno, Surya},
title = {Efficient Transformer Models via Language-Aware
Frequency-Based Vocabulary Pruning},
journal = {IEEE Access},
volume = {14},
pages = {50993--51006},
year = {2026},
doi = {10.1109/ACCESS.2026.3679735}
}