ReVonance Support Drafter (DPO, full)
The Drafter agent of a LangGraph multi-agent support system for the fictional shop ReVonance.
Base model Qwen/Qwen2.5-0.5B-Instruct, fine-tuned with full fine-tuning.
Stage: dpo (sft = supervised on the response simulator; dpo = then aligned with human and
critic preferences via Direct Preference Optimisation).
Evaluation (held-out simulated tickets, full multi-agent graph)
| model | A | B | T | approval A | approval B | Δ (B − A, pts) | B only | A only | McNemar p |
|---|---|---|---|---|---|---|---|---|---|
| qwen2.5-0.5b | LoRA SFT | Full SFT | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
| qwen2.5-0.5b | LoRA SFT | Full SFT | 0.7 | 1 | 1 | 0 | 0 | 0 | 1 |
| qwen2.5-0.5b | LoRA SFT → LoRA DPO | Full SFT → LoRA DPO | 0 | 0.832 | 1 | 16.8 | 28 | 0 | 0 |
| qwen2.5-0.5b | LoRA SFT → LoRA DPO | Full SFT → LoRA DPO | 0.7 | 0.832 | 0.976 | 14.4 | 28 | 4 | 0 |
| qwen2.5-0.5b | LoRA SFT → LoRA DPO | Full SFT → full DPO | 0 | 0.832 | 0.862 | 3 | 20 | 15 | 0.4996 |
| qwen2.5-0.5b | LoRA SFT → LoRA DPO | Full SFT → full DPO | 0.7 | 0.832 | 0.874 | 4.2 | 26 | 19 | 0.3713 |
| qwen2.5-0.5b | LoRA SFT | LoRA SFT → LoRA DPO | 0 | 1 | 0.832 | -16.8 | 0 | 28 | 0 |
| qwen2.5-0.5b | LoRA SFT | LoRA SFT → LoRA DPO | 0.7 | 1 | 0.832 | -16.8 | 0 | 28 | 0 |
| qwen2.5-0.5b | Full SFT | Full SFT → LoRA DPO | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
| qwen2.5-0.5b | Full SFT | Full SFT → LoRA DPO | 0.7 | 1 | 0.976 | -2.4 | 0 | 4 | 0.125 |
| qwen2.5-0.5b | LoRA SFT seed 7 | LoRA SFT seed 8 | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
| qwen2.5-0.5b | LoRA SFT seed 7 | LoRA SFT seed 8 | 0.7 | 1 | 1 | 0 | 0 | 0 | 1 |
| qwen2.5-0.5b | Full SFT seed 7 | Full SFT seed 8 | 0 | 1 | 1 | 0 | 0 | 0 | 1 |
| qwen2.5-0.5b | Full SFT seed 7 | Full SFT seed 8 | 0.7 | 1 | 0.994 | -0.6 | 0 | 1 | 1 |
Knowledge-base fingerprint: f9cdb07b5df6. Demo model trained on synthetic data; it only
knows the toy policies of ReVonance.
Pipeline: Full SFT → LoRA DPO (Lab 3 winner).