ReVonance Support Drafter (DPO, lora)
The Drafter agent of a LangGraph multi-agent support system for the fictional shop ReVonance.
Base model HuggingFaceTB/SmolLM2-360M-Instruct, fine-tuned with LoRA (merged).
Stage: dpo (sft = supervised on the response simulator; dpo = then aligned with human and
critic preferences via Direct Preference Optimisation).
Evaluation (held-out simulated tickets, full multi-agent graph)
| stage | method | temperature | first-draft approval | fda 95% CI | resolved | avg drafts |
|---|---|---|---|---|---|---|
| base | none | 0 | 0 | 0.00–0.02 | 0.072 | 2.94 |
| sft | lora | 0 | 0.994 | 0.97–1.00 | 0.994 | 1.012 |
| sft | lora | 0.7 | 0.988 | 0.96–1.00 | 0.988 | 1.024 |
| dpo | lora | 0 | 1 | 0.98–1.00 | 1 | 1 |
| dpo | lora | 0.7 | 0.982 | 0.95–0.99 | 0.982 | 1.036 |
Knowledge-base fingerprint: f9cdb07b5df6. Demo model trained on synthetic data; it only
knows the toy policies of ReVonance.
Preferences: {'synthetic': 400, 'synthetic (A/B, critic)': 80} (β=0.1, epochs=2).