FastAPI Assistant: LoRA adapter (v2026.11.14-72d6b37)
A LoRA fine-tune of Qwen/Qwen2.5-Coder-1.5B-Instruct that answers questions about the FastAPI web framework. This repository holds the LoRA adapter; load it on top of the base model.
The tag v2026.11.14-72d6b37 is the release; pin it with revision= to get exactly these weights.
Usage
With transformers and peft, on top of the base model:
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Coder-1.5B-Instruct")
model = PeftModel.from_pretrained(model, "syed-waleed-ahmed/fastapi-assistant-lora", revision="v2026.11.14-72d6b37")
messages = [
{"role": "system", "content": "You are a senior Python engineer specialized in the FastAPI web framework. Answer questions concisely and correctly, and include minimal runnable code examples where they help. If a question is outside FastAPI, say so briefly."},
{"role": "user", "content": "How do I return a 201 status code from a POST endpoint?"},
]
inputs = tokenizer.apply_chat_template(
messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
)
output = model.generate(**inputs, max_new_tokens=256, do_sample=False)
text = tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True)
for marker in ["<|im_end|>", "<|im_start|>", "<|user|>", "<|system|>", "<|assistant|>"]: # see Prompt format
text = text.split(marker)[0]
print(text.strip())
To serve it as an HTTP API with streaming, retrieval over the FastAPI documentation and an OpenAI-compatible endpoint, use the source repository.
Prompt format
Use the system prompt the model was trained with, in the chat template of the base model (stored in the GGUF and in the base model's tokenizer):
You are a senior Python engineer specialized in the FastAPI web framework. Answer questions concisely and correctly, and include minimal runnable code examples where they help. If a question is outside FastAPI, say so briefly.
The template's role markers (<|im_end|>, <|im_start|>, <|user|>, <|system|>, <|assistant|>) are plain text to this model, not special tokens. When it does not end its answer with the end-of-sequence token, it runs on into an invented next turn, so stop generation at any of them, as the example does.
Evaluation
The release gate answers 108 held-out FastAPI questions with greedy decoding and scores each answer for keyword recall (the API names a correct answer has to mention), code validity (its Python blocks parse) and ROUGE-L against a reference answer. The composite score is their mean per question. Scored with transformers, with the unquantized model on the training GPU; the quantized GGUF is scored again with llama.cpp before it is deployed.
| Metric | This release | Base model |
|---|---|---|
| Composite score | 0.672 | 0.558 |
| Keyword recall | 0.711 | 0.622 |
| Code validity | 0.856 | 0.721 |
| ROUGE-L | 0.455 | 0.345 |
The metrics are automatic and lexical: they reward naming the right APIs and code that parses, not correctness. The model card in the source repository has the scores of both release gates, results per category and examples of wrong answers.
Training
- LoRA with completion-only loss (only answer tokens are trained), using the same chat template in training, evaluation and serving
- LoRA r=16, alpha=32, dropout=0.05; target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
- Epochs=3, lr=0.0002, max_seq_length=512
- Data: the curated FastAPI question-answer pairs in
data/; the held-out evaluation set is never used for training or checkpoint selection
Intended use and limitations
Short answers to FastAPI questions, each with a minimal code example, for developers who can check them.
- At 1.1B parameters it is often wrong, including in code that parses: it invents parameters and APIs and states them confidently. Check answers against the FastAPI documentation.
- Generated code is untrusted. Review it, and run it only in a sandbox.
- FastAPI and English only, with a 2,048-token context window.
License and attribution
Derived from Qwen/Qwen2.5-Coder-1.5B-Instruct, released by its authors under the
Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0). This repository
contains modified work: a LoRA adapter trained on top of it. It is distributed under the same
license; see NOTICE.