Qwen3-1.7B Depth-Aggressive — Q4_K_M GGUF
A structured depth-pruned and recovery-trained variant of
Qwen/Qwen3-1.7B, converted to GGUF
and quantized as Q4_K_M for llama.cpp-compatible runtimes.
Model changes
- Transformer layers: 28 → 24
- Removed source layers: 9, 14, 19, 7
- Parameters: approximately 1.721B → 1.519B
- Parameters retained: 88.30%
- Recovery: 100 chat-recovery optimizer steps
- Recovery objective: assistant-only cross-entropy plus teacher distillation
- GGUF quantization: Q4_K_M
File
| File | Quantization | Size | SHA-256 |
|---|---|---|---|
Qwen3-1.7B-Depth-Aggressive-Q4_K_M.gguf |
Q4_K_M | 940.82 MiB | eb836a316bad3d3672d0e182533f8cb306f5783ae68d9fdb8955c5d2fb1184ae |
llama.cpp
llama-cli -m Qwen3-1.7B-Depth-Aggressive-Q4_K_M.gguf -cnv -ngl 99
Notes
The pruning experiment showed the aggressive depth-pruned 1.7B checkpoint at 24 layers and about 1.519B parameters before GGUF quantization. This repository contains the quantized GGUF build, not the full-precision Transformers weights.
License
The base model is distributed under the Apache 2.0 license. Review the original Qwen model repository for its complete terms and documentation.