Qwen2.5-3B JSON Structured Output (LoRA)
A LoRA adapter that enhances Qwen2.5-3B-Instruct for extracting structured JSON from unstructured text.
Training Details
| Parameter | Value |
|---|---|
| Base Model | Qwen/Qwen2.5-3B-Instruct |
| Method | QLoRA (4-bit NF4) |
| LoRA Rank | 16 |
| LoRA Alpha | 32 |
| Target Modules | q,k,v,o,gate,up,down |
| Trainable Params | 29.9M (1.76%) |
| Training Examples | 20 curated |
| Epochs | 5 |
| Learning Rate | 5e-5 |
| Final Loss | 1.506 |
| Hardware | NVIDIA RTX 3090 x2 |
Quick Start
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch, json
bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, "2reb/Qwen2.5-3B-JSON-StructuredOutput")
License
MIT