Messy2JSON - Qwen2.5-0.5B
A 0.5B parameter model fine-tuned with QLoRA to convert messy text into structured JSON.
Training
- Base model: Qwen/Qwen2.5-0.5B-Instruct
- Method: QLoRA (4-bit NF4 + LoRA r=16, alpha=32)
- Dataset: stindardlogic/data-extraction-sft-100k (39k JSON-filtered samples, 9k used for training)
- Epochs: 2
- Final validation loss: 0.0045
- Token accuracy: 99.82%
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_id = "aradmanamnaoon/messy2json-qwen-0.5b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float32)
messages = [
{"role": "system", "content": "You extract structured data from messy text and output ONLY valid JSON."},
{"role": "user", "content": "Best regards, Jane Doe, VP Engineering at Acme, jane@acme.com"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
Limitations
- Field names are not fixed. The model improvises JSON keys based on input. Two similar inputs may produce different schemas.
- Occasional nesting quirks. Fields may be nested under semantically close parents.
- Domain sensitivity. Best on invoices, email signatures, compliance excerpts, meeting notes, and log lines.
- No schema enforcement. The model infers structure from the text.
- 0.5B parameter model. Limited reasoning.
- English only.
Recommended Use
- Rapid prototyping of text-to-JSON pipelines
- Pre-processing messy documents before downstream parsing
- Edge deployment where a 1 GB model is preferred over an API call
Not Recommended For
- Production systems requiring guaranteed schema compliance
- Non-English text
- Inputs requiring multi-document reasoning
- Safety-critical extraction without human review
Training Details
| Parameter | Value |
|---|---|
| Base model | Qwen/Qwen2.5-0.5B-Instruct |
| Method | QLoRA (4-bit NF4 + LoRA) |
| LoRA rank (r) | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Trainable parameters | ~4M of ~498M (0.85%) |
| Epochs | 2 |
| Learning rate | 2e-4 |
| Max sequence length | 1024 |
| Training samples | 9,000 |
| Validation samples | 1,000 |
| Training time | ~40 minutes on a T4 GPU |
| Final validation loss | 0.0045 |
| Mean token accuracy | 99.82% |
Dataset
Fine-tuned on stindardlogic/data-extraction-sft-100k, filtered to JSON-only outputs.
Ethical Considerations
- Privacy: Do not feed PII, credentials, or confidential data into a public Space demo.
- Hallucination: The model may invent fields or values not present in the input.
- No provenance tracking. Output JSON does not cite which part of the input each field came from.
Citation
@misc{messy2json-qwen-0.5b,
title = {Messy2JSON: Qwen2.5-0.5B fine-tuned for structured extraction},
author = {aradmanamnaoon},
year = {2025},
url = {https://huggingface.co/aradmanamnaoon/messy2json-qwen-0.5b}
}
Contact
Open a discussion on the model page.
Acknowledgments
- Base model by Qwen team
- Training framework: Hugging Face TRL, PEFT, Transformers
- Dataset: stindardlogic/data-extraction-sft-100k
- Compute: Google Colab free T4 GPU
License
Apache 2.0 - same as the base model. See LICENSE for details.
You are free to use, modify, and redistribute this model, including for commercial purposes, provided you retain the license notice.
Model Card Version
- v1.0 - initial release