This is a LoRA add-on for Qwen/Qwen3-4B, trained to answer questions about French administrative procedures from passages of the official Service-Public.fr guides that it is given. It writes a JSON reply with:
It is the fine-tuning experiment of Démarches FR, a local RAG assistant with a measured evaluation. On its own, without the project's search and prompt, it is not meant to answer anything.
| model | right behaviour | facts found | answers with invented content |
|---|---|---|---|
| Qwen3-4B, before fine-tuning (LLM judge) | 73.6% | 72.3% | 32.5% |
| this add-on (LLM judge) | 79.8% | 78.7% | 30.1% |
| this add-on (reference grading) | 85.9% | 76.3% | 27.0% |
| Qwen3-8B + reasoning, the project's final model (reference grading) | 92.6% | 81.4% | 15.3% |
Reference grading: every answer read against its passages by a stronger model (Claude); the LLM judge is Qwen3-8B. Full method and results: finetune/README.md.
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-4B")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B", device_map="auto")
model = PeftModel.from_pretrained(model, "youssef-khazal/qwen3-4b-demarches-lora")
It expects the prompt built by Assistant.build_messages in the project, with thinking off (enable_thinking=False).
The training prompts contain passages of "Fiches pratiques et ressources de Service-Public.gouv.fr Particuliers" (DILA), under the Licence Ouverte / Open Licence 2.0. Source : Service-Public.fr / DILA — mise à jour : 25/10/2026.
Qwen/Qwen3-4B