加载中…
加载中…
MAAI 902 project – Navi Dharshan B. Fine-tuned on 3683 train / 460 validation / 460 test redacted Boston FIO narratives (2024) (Analyze Boston, PDDL licence; names, officer names, addresses and neighbourhoods replaced with placeholders).
Training: learning rate 0.0001 with 10% warm-up and linear decay, batch 8, weight decay 0.01, block size 128; early stopping on validation loss (patience 3) stopped at epoch 4.74; best checkpoint at epoch 3.99 (validation loss 2.276).
Held-out test set
| Metric | Base DistilGPT-2 | Fine-tuned |
|---|---|---|
| Test perplexity (lower = better) | 78.617 | 10.016 |
| Next-token accuracy, top-1 | 0.309 | 0.529 |
| Next-token accuracy, top-5 | 0.486 | 0.747 |
| ROUGE-L F1 | 0.100 | 0.179 |
| ROUGE-L precision | 0.099 | 0.192 |
| ROUGE-L recall | 0.106 | 0.177 |
| BLEU-4 | 0.004 | 0.038 |
| Domain-style rate | 0.027 | 0.987 |
| Repetition rate (lower = better) | 0.000 | 0.004 |
| Distinct-2 (variety) | 0.597 | 0.322 |
All generated text is synthetic, describes no real event and must not be presented as a police record.
distilbert/distilgpt2