Qwen3.5-4B Q4_K_M (imatrix) - Lynn calibration
This repository ships an imatrix-calibrated Q4_K_M GGUF of the official Qwen/Qwen3.5-4B BF16 weights, built on DGX Spark (GB10, sm_121) by the Lynn team.
This is a pure quantization of the upstream model. It is not a distillation and not a Lynn-native NVFP4/W4A8 checkpoint.
Files
| File | Size | SHA256 | Role |
|---|---|---|---|
Qwen3.5-4B-Q4_K_M-imatrix.gguf |
2.6 GB | 7abaf02bbe25c608deb308db526766f761ad4fb85c512a69ff36520c4b304b23 |
llama.cpp / Ollama / LM Studio GGUF weights |
Qwen3.5-4B.imatrix |
3.5 MB | 863a93c58a14925b58303a369d9bb155411b40d52fde121f195eaf7a6691f07c |
imatrix calibration data used for quantization |
Build Details
- Source weights:
Qwen/Qwen3.5-4Bofficial BF16 - Converter:
llama.cpp convert_hf_to_gguf.py --outtype f16 - Calibration: Lynn Chinese + English + code mix, 256 chunks, 512 ctx
- Quantizer:
llama-quantize --imatrix Qwen3.5-4B.imatrix ... Q4_K_M - Built on: DGX Spark (GB10, sm_121), 2026-05-24
Evaluation
V8/V9, MMLU500, and GPQA Diamond thinking-on evaluations are running on Spark. Those artifacts and scores will be added in a follow-up update.
Run
llama-server \
-m Qwen3.5-4B-Q4_K_M-imatrix.gguf \
--host 0.0.0.0 --port 18099 \
--ctx-size 32768 --n-gpu-layers 999 \
--jinja --reasoning on
The GGUF embeds the upstream Qwen3.5 chat template, including thinking-mode
support via chat_template_kwargs.enable_thinking.
License
Apache 2.0, inherited from Qwen/Qwen3.5-4B.