🧠 Introduction
Qwen2.5-0.5B-Distilled-from-1.5B-AlpacaChinese-v1 is a distilled Chinese-language model based on the Qwen2.5-0.5B architecture. It was trained using Qwen2.5-1.5B-Instruct as the teacher model and the Alpaca-Chinese-5K-Distill-Qwen dataset, leveraging both soft labels (KL Loss) and hard labels (Cross-Entropy Loss) in a joint distillation framework to balance performance and efficiency.
💡 Model Highlights
- 🚀 Lightweight & Efficient: With only 0.5 billion parameters, it's ideal for edge devices and local deployment.
- 🧪 Joint Distillation Optimization: Combines soft and hard targets to preserve teacher model capabilities.
- 📚 Enhanced Chinese Proficiency: Covers a wide range of tasks, including Q&A, reasoning, and writing.
- 📱 Quantization-Ready: Supports
q8_0 quantization and runs smoothly on iPhone and Mac.
🏗️ Training Configuration
📊 Chinese C-Eval Benchmark
📱 Deployment with llama.cpp (q8_0)
🧠 简介
Qwen2.5-0.5B-Distilled-from-1.5B-AlpacaChinese-v1 是一个中文领域蒸馏模型，基于 Qwen2.5-0.5B 架构，以 Qwen2.5-1.5B-Instruct 为教师，使用 Alpaca-Chinese-5K-Distill-Qwen 数据集进行软标签 KL Loss + 硬标签 CE 的联合蒸馏优化，兼顾性能与效率。
💡 模型亮点
- 🚀 轻量高效：仅 0.5B 参数，适合边缘设备与本地部署。
- 🧪 联合蒸馏优化：结合软硬标签，保留教师模型能力。
- 📚 中文能力增强：覆盖问答、推理、写作等多种中文任务。
- 📱 原生量化适配：支持 q8_0 量化，已实测 iPhone 与 Mac 运行流畅。
🏗️ 训练配置
📊 中文 C-Eval 评测
📱 llama.cpp q8_0 部署测试
🔧 Quick Usage
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("eggbiscuit/Qwen2.5-0.5B-Distilled-from-1.5B-AlpacaChinese-v1", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("eggbiscuit/Qwen2.5-0.5B-Distilled-from-1.5B-AlpacaChinese-v1", trust_remote_code=True).cuda()
prompt = "请写一首关于春天的七言诗。"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
📚 Citation
@misc{qwen2.5,
author = {Qwen Team},
title = {Qwen Technical Report},
year = 2024,
howpublished = {\url{https://arxiv.org/abs/2412.15115}}
}
@misc{alpaca_chinese,
title={Alpaca-Chinese Dataset},
author={Yiming Cui et al.},
howpublished = {\url{https://github.com/ymcui/Chinese-LLaMA-Alpaca}}
}
🔗 Related Links
