Continued next-token pretraining of Qwen/Qwen2.5-0.5B with LoRA, merged back into the base
weights. The receiver model was not involved in this training run.
| model | documents | tokens | mean loss | perplexity |
|---|---|---|---|---|
| original donor (Qwen2.5-0.5B) | 200 | 159280 | 3.1133 | 22.495 |
| trained donor | 200 | 159280 | 3.3854 | 29.529 |
| original receiver | 200 | 159280 | 3.1596 | 23.562 |
| transferred receiver | 200 | 159280 | 3.1673 | 23.744 |
Perplexity measures language-model fit on held-out documents of the same corpus. It is not factual accuracy.
Corpus: newsdata01/ai-agents-autonomous-systems-news-dataset (mit), documents dated 2026-10-01 .. 2026-10-30. Only the article bodies are used for training; the dataset itself is not redistributed here. Article copyright remains with the original publishers.
The corpus carries no shared event identifiers, so the train/eval split is document-level: two articles covering the same event can appear on opposite sides. This is weaker than an event-cluster split and must be stated in any report.
Released under the licence of the base checkpoint (Qwen2.5: Apache-2.0). Review the upstream model card before redistribution or commercial use.
Qwen/Qwen2.5-0.5B