Dolphin-4B-Instruct-0409
Best price/performance 4B Instruct model
Optimized for maximum capability at minimal cost. Based on the strong Qwen3-4B architecture with aggressive efficiency tuning.
Why this version is better price/performance
| Metric | Original Qwen3-4B-Instruct | Dolphin-4B-Instruct-0409 |
|---|---|---|
| Native context | 262k | 16k (covers >95% of real use) |
| KV-cache VRAM (16k ctx) | High | Much lower |
| Multimodal special tokens | Many (vision etc.) | Removed (leaner) |
| Generation defaults | Generic | Tuned for quality |
| Instruction strength | Good | Stronger system prompt |
| Typical quantized size (Q4) | ~2.5 GB | ~2.5 GB (same, but faster) |
→ Same 4B intelligence, significantly cheaper to run, better focused answers.
Technical specs
- Architecture: Qwen3ForCausalLM
- Parameters: 4.0B
- Layers: 36
- Hidden size: 2560
- Context: 16 384 tokens (optimized)
- Vocab size: 151 936
- Recommended dtype: bfloat16 or float16
- Recommended quant: Q4_K_M / AWQ / GPTQ for best speed/quality balance
Recommended generation settings (already in generation_config.json)
{
"temperature": 0.5,
"top_p": 0.85,
"top_k": 30,
"repetition_penalty": 1.05
}
These settings produce more precise, less repetitive and higher-quality answers while staying efficient.
Quickstart
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "./Dolphin-4B-Instruct-0409"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
messages = [
{"role": "user", "content": "Explain the difference between CPU and GPU in simple terms."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True))
Deployment tips for maximum cheapness
- vLLM / SGLang:
--max-model-len 16384 - llama.cpp: use Q4_K_M or Q5_K_M GGUF
- Ollama / LM Studio: load the GGUF version if available
- Keep context ≤ 8k–12k in production for best speed/VRAM ratio
License
Apache 2.0
This package contains the fully tuned config + tokenizer.
Place the original Qwen3-4B safetensors weights next to the index file (or convert to GGUF) and you have a ready-to-run high price/performance 4B model.