Model Name: Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1
Base Model: Qwen3.5-9B
Training Type: Supervised Fine-Tuning (SFT, Distillation)
Parameter Scale: 9B
Training Framework: Unsloth
This model is a distilled variant of Qwen3.5-9B, trained on high-quality reasoning data derived from GLM-5.1.
The primary goals are to:
Jackrong/GLM-5.1-Reasoning-1M-CleanedKassadin88/GLM-5.1-1000000x dataset.Qwen3.5-reasoning-700xJackrong/Qwen3.5-reasoning-700x[!IMPORTANT] Training used
Jackrong/GLM-5.1-Reasoning-1M-Cleaned, a cleaned derivative ofKassadin88/GLM-5.1-1000000x. Special thanks to Kassadin88 ❤️ for the original dataset. Please support the original author with a follow and a like. Only a quality-filtered subset was used for distillation, rather than the full original dataset.
Base Model (Qwen3.5-9B)
│
▼
Qwen3.5-9B fine-tuned with Unsloth
│
▼
Supervised Fine-Tuning (SFT) + LoRA
Distillation from GLM-5.1 reasoning data
│
▼
Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1
This model learns a reasoning structure distilled from GLM-5.1 traces, rather than the previous Qwopus / Claude-style scaffold.
From the GLM-5.1 distillation data, the reasoning pattern is usually more task-first and structure-driven:
A typical abstract scaffold looks like:
Example:
The user is asking about [Topic / Problem] under [Specific Constraints].
This is mainly a [reasoning / coding / math / STEM / instruction-following] task.
Understand the task
Break down the problem
Reason step by step
Construct the final answer
[!NOTE] Compared with the previous Claude-style reasoning scaffold, this GLM-5.1 distillation data is more focused on structured task decomposition, domain-aware reasoning, and final-answer organization.
For a 9B student model, the goal is not to copy the teacher perfectly, but to learn a cleaner reasoning procedure and produce more stable outputs.
Compared to typical SFT datasets:
This model is intended to deliver incremental but meaningful improvements in practical use:
[!WARNING] For 9B-scale models, gains from SFT are typically gradual rather than dramatic. The main benefit is usually better consistency, clearer reasoning, and stronger answer organization, rather than a sudden jump in raw capability.
This model treats distillation as more than simple output imitation.
The goal is not to make a 9B model copy the teacher token by token, but to transfer a stronger reasoning structure and problem-solving style into Qwen3.5-9B.
In this project, high-quality teacher data is valuable because it provides:
[!NOTE] High-quality reasoning supervision can help the student model better use its existing knowledge, rather than simply replacing it with teacher outputs.
Recent work:
Ren et al., 2026 — Rethinking Generalization in Reasoning SFT (arXiv:2604.06628)
Short-epoch reasoning SFT can underestimate generalization — in-domain gains may appear early, while out-of-domain improvements often require sufficient optimization.
This paper shows that generalization in reasoning SFT is not fixed, but conditional — depending on optimization, data quality, and model capability.
Key takeaways:
For this project, that evidence matters because it supports a more patient interpretation of distillation-style SFT. If reasoning supervision is clean and sufficiently optimized, the resulting gain is not necessarily immediate or linear, but it can still be real and transferable.
This aligns closely with the philosophy of this release:
[!IMPORTANT] This suggests that the improvement is not simply memorization or dataset overlap. Instead, sufficiently optimized reasoning SFT can help the student model:
- 🧠 Better utilize existing knowledge
- 🔍 Activate latent knowledge through structured reasoning
- 🏗️ Learn reasoning procedures, not just output format
👉 GitHub Repository: Jackrong-llm-finetuning-guide Visit the repo to dive into the codebase and reproduce the results locally or on Colab.
🔗 Qwopus3.5-27b Complete Fine-Tuning Guide (PDF)
A Note: My goal isn't just to detail a workflow, but to demystify LLM training. Beyond the social media hype, fine-tuning isn't an unattainable ritual—often, all you need is a Google account, a standard laptop, and relentless curiosity. All training and testing for this project were self-funded. If you find this model or guide helpful, a Star ⭐️ on GitHub would be the greatest encouragement. Thank you! 🙏
This project would not have been possible without the support and contributions of the open-source community.
Special thanks to the Unsloth AI team for making efficient fine-tuning of large language models more accessible. This qwen3_5 model was trained with Unsloth and Hugging Face's TRL library, enabling a significantly faster and more practical fine-tuning workflow.
I would also like to acknowledge:
GLM-5.1-1000000x dataset that this training pipeline ultimately builds upon.Jackrong/GLM-5.1-Reasoning-1M-Cleaned for making the source data more consistent and practical for distillation training.If you use this model in your research or projects, please cite:
@misc{jackrong_qwen35_9b_glm51_distill_v1,
title = {Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1},
author = {Jackrong},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/Jackrong/Qwen3.5-9B-GLM5.1-Distill-v1}}
}