Qwen2.5-0.5B grounded SFT
Supervised fine-tune of Qwen/Qwen2.5-0.5B-Instruct on the preferred answers in the grounded QA preference dataset.
This is the policy that initializes the RLOO stage. It is trained with LoRA on the chosen responses only, then the adapter is merged into the base weights.
The training data is derived from SQuAD 2.0 and is licensed CC BY-SA 4.0. These weights stay under the base model's Apache 2.0 license.
Intended use
Answer a question from a passage the user provides. If the passage does not contain the answer, reply with exactly: The passage does not say.
Training
- Trainer: TRL
SFTTrainer - Loss: completion tokens only
- LoRA rank 16, merged before upload
- The config file that produced these weights is named in the metrics section below
Evaluation
Greedy decoding. Strict accuracy requires the gold span in a short reply that does not abstain, or exactly The passage does not say. The table also reports normalized answer EM, token F1, and semantic abstention. Final numbers for a protocol run are the article-held-out test split. resolved.yaml and environment.json record the configuration and package versions.
Config: protocol.yaml.
| Model | Strict acc. | Answerable | Unanswerable | Answer EM | Answer F1 | Semantic abstention |
|---|---|---|---|---|---|---|
| base | 0.410 | 0.290 | 0.530 | 0.200 | 0.290 | 0.530 |
| always_abstain | 0.500 | 0.000 | 1.000 | 0.000 | 0.000 | 1.000 |
| sft | 0.550 | 0.310 | 0.790 | 0.300 | 0.346 | 0.790 |
| rloo | 0.475 | 0.590 | 0.360 | 0.520 | 0.616 | 0.360 |
| Comparison | Reward-model win rate | Ties |
|---|---|---|
| rloo_vs_sft | 0.410 | 0.565 |
| rloo_vs_base | 0.510 | 0.340 |
| rloo_vs_always_abstain | 0.705 | 0.260 |
| sft_vs_base | 0.285 | 0.480 |
| sft_vs_always_abstain | 0.315 | 0.670 |
Limitations
This is a 0.5B research model. It is not a general assistant, and it was not trained with new human preference labels. It will miss answers, abstain when it should not, and copy a span when the passage does not support it.