Nasim435/Qwen-3B-Automotive-10k

QWEN-3B MODEL CARD / AUTOMOTIVE DOMAIN ■ ONLINE QWEN-3B
AUTOMOTIVE BASE → Qwen/Qwen2.5-3B-Instruct  ·  METHOD → QLoRA / 4-bit NF4  ·  SAMPLES → 10,000 QLoRA Automotive PEFT SFT Unsloth 4-bit NF4 bfloat16 TRL BitsAndBytes MIT License 01 OVERVIEW

Domain-adapted variant of Qwen2.5-3B-Instruct, fine-tuned on automotive instruction-following data using QLoRA with Unsloth optimization. Built as an experimental research project focused on modern PEFT workflows and efficient LLM engineering.

Specialized toward automotive question answering, diagnostic explanations, vehicle maintenance assistance, and technical guidance. Trained on a curated subset of 10,000 samples from the BAAI automotive industry instruction dataset.

Model Identity BASEQwen/Qwen2.5-3B-Instruct DEVELOPED BYAlibaba Cloud / Qwen Team FINE-TUNED BYNasim Raj Laskar LANGUAGEEnglish LICENSEMIT Capabilities ► Automotive Q&A ► Diagnostic Explanations ► Repair & Maintenance Guidance ► Vehicle Systems Knowledge ► Troubleshooting Instructions 02 TRAINING DATA

Fine-tuned on the English subset of BAAI/IndustryInstruction_Automobiles. A custom subset of approximately 10,000 automotive instruction samples was selected for training.

Dataset Processing Pipeline FORMATQwen chat-template conversations STRUCTUREsystem → user → assistant SYSTEM PROMPT"You are an automotive expert assistant." SAMPLES~10,000 curated instruction pairs 03 TRAINING CONFIGURATION MethodQLoRA Quantization4-bit NF4 Precisionbfloat16 OptimizerAdamW 8bit Max Seq Len512 PackingDisabled ► LoRA Adapter Parameters Rank (r)16 Alpha (α)32 Dropout0.05 Trainable Params~30M Total Params~3.1B Infrastructure PLATFORMAWS SageMaker GPUNVIDIA L4 Libraries Transformers · TRL · PEFT · BitsAndBytes Accelerate · Datasets · Unsloth ◆ UNSLOTH OPTIMIZATIONS

Integrated Unsloth optimizations reduced model download size from ~6GB to ~3GB. Delivered stable low-memory 4-bit training, high GPU utilization, and accelerated transformer patching throughout the QLoRA workflow.

04 EVALUATION RESULTS

EVALUATED ON 100 SAMPLES

Model Performance PERPLEXITY5.63 BLEU SCORE46.9% SIMILARITY SCORE16.7% Inference Metrics AVG LATENCY2779 ms THROUGHPUT18 tok/sec GPU Metrics PEAK VRAM7.28 GB AVG GPU UTIL80.3% MAX GPU UTIL99.0% Training Throughput AVG TRAINING540 tok/sec MAX TRAINING2104 tok/sec 05 EXAMPLE USAGE Python

# Load model from Hugging Face Hub
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name = "Nasim435/Qwen-3B-Automotive-10K"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model     = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto"
)
prompt   = "Explain symptoms of a failing alternator."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
inputs  = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=150,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

06 LIMITATIONS ⚠ WARNING

• Experimental fine-tuned model — not intended for production safety systems
• May hallucinate or generate inaccurate automotive advice
• Not suitable for safety-critical or professional mechanical decision-making
• Trained on ~10k samples; generalization may be limited

07 ACKNOWLEDGEMENTS Qwen TeamAlibaba Cloud BAAIDataset Contributors Hugging FaceEcosystem TRL / PEFTContributors UnslothContributors BitsAndBytesQuantization AccelerateFramework Open-SourceLLM Community NASIM RAJ LASKAR Qwen-3B-Automotive-10K
MIT License · English · 2026