TinySQL-3B is a specialized, lightweight text-to-SQL model fine-tuned from Qwen/Qwen2.5-3B-Instruct using 4-bit QLoRA. It is tailored to translate natural language analytics queries into performant, syntactically accurate PostgreSQL 16 queries against the 9-table Brazilian E-Commerce (Olist) database.
Qwen/Qwen2.5-3B-Instruct (3.09 Billion Parameters)q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj)TRL (SFTTrainer)SELECT) and refusing destructive actions (DROP, DELETE, UPDATE, INSERT).REFUSE when user requests refer to data absent from the schema (e.g. ad spend, margins, customer salary).EXTRACT(DOW ...), DATE_TRUNC, ILIKE). Generating SQLite or Oracle SQL is not guaranteed.orders (99,441 rows)order_items (112,650 rows)customers (99,441 rows)products (32,951 rows)sellers (3,095 rows)order_payments (103,886 rows)order_reviews (99,224 rows)category_translation (71 rows)geolocation (1,000,163 rows)| Parameter | Value |
|---|---|
| Precision | 4-bit NormalFloat (NF4), bfloat16 adapter compute |
| LoRA Rank ($r$) | 16 |
| LoRA Alpha ($\alpha$) | 32 |
| LoRA Dropout | 0.0 (Unsloth optimized) |
| Optimizer | adamw_8bit |
| Learning Rate | 2e-4 |
| LR Schedule | Cosine with 5 warmup steps |
| Effective Batch Size | 8 (Per-device batch 2 $\times$ Gradient Accumulation 4) |
| Epochs / Steps | 3 Epochs (60 steps) |
| Max Sequence Length | 2048 tokens |
| Compute Hardware | 1x NVIDIA Tesla T4 (16 GB VRAM) on Kaggle |
| Training Duration | 14 minutes 12 seconds |
| Final Train Loss | 0.2984 |
| Final Eval Loss | 0.1140 |
Scored on Execution Accuracy against live PostgreSQL 16 with deterministic SHA-256 result hashing:
| Category | Questions | Zero-Shot Raw Accuracy | Self-Healing Agent Accuracy |
|---|---|---|---|
| Single-Table Aggregations | 13 | 100.0% (13/13) | 100.0% (13/13) |
| Refusal Precision (Unanswerable) | 3 | 100.0% (3/3) | 100.0% (3/3) |
| Date & Timestamp Arithmetic | 3 | 33.3% (1/3) | 66.7% (2/3) |
| Multi-Table Joins | 8 | 25.0% (2/8) | 75.0% (6/8) |
| Analytical / Window Functions | 3 | 0.0% (0/3) | 66.7% (2/3) |
| Overall | 30 | 63.3% (19/30) | 86.7% (26/30) |
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_name = "Qwen/Qwen2.5-3B-Instruct"
adapter_path = "train/tinysql_adapter"
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
torch_dtype=torch.float16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_path)
prompt = """<|im_start|>system
You are TinySQL, a PostgreSQL 16 Text-to-SQL expert. Return ONLY valid SQL.<|im_end|>
<|im_start|>user
What are the top 3 product categories by delivered revenue?<|im_end|>
<|im_start|>assistant
"""
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
docker run --gpus all -p 8000:8000 \
-v ./train/tinysql_adapter:/models/tinysql \
vllm/vllm-openai:latest \
--model /models/tinysql \
--served-model-name tinysql \
--max-model-len 2048 \
--gpu-memory-utilization 0.9
adapter_model.safetensors 120 MBcheckpoint-60/adapter_model.safetensors 120 MBcheckpoint-60/optimizer.pt 61 MBcheckpoint-60/rng_state.pth 14.6 kBcheckpoint-60/training_args.bin 6.1 kBcheckpoint-60/scheduler.pt 1.5 kBcheckpoint-60/scaler.pt 1.4 kBQwen/Qwen2.5-3B-Instruct