DistilBERT fine-tuned on operating systems text
distilbert-base-uncased continued-pretrained with masked language modeling on text about
operating systems (processes, scheduling, deadlocks, memory management, file systems).
| Perplexity (lower is better) | |
|---|---|
| Before fine-tuning | 19.1 |
| After fine-tuning | 5.0 |
Trained on a small corpus as a learning project, so expect modest gains.
Use it with: pipeline("fill-mask", model="aditya500/distilbert-os-fill-mask")