Qwen3.8-27B Q5_K_M GGUF

This repository contains one independently downloadable Q5_K_M GGUF of Qwen/Qwen3.8-27B. It is an unmodified vanilla quantization, not a fine-tune, merge, ablation, alignment change, or chat-template modification.

The source weights are pinned to 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0. The official checkpoint's internal Qwen3_5ForConditionalGeneration / qwen3_5 architecture identifier does not mean this is a Qwen3.5 model.

Files and conversion

• Main model: Qwen3.8-27B-Q5_K_M.gguf (19.536 GB decimal)
• Vision projector: mmproj-Qwen3.8-27B-F16.gguf
• Complete repository size: 20.476 GB decimal
• Quantization: Q5_K_M using llama.cpp K/IQ quantization
• llama.cpp revision: 5f754ea0e2fd21e1213db7ebebfd65d938d9d69c
• Calibration: none for K-quants; representative local prompts where required by IQ conversion
• Source inventory: 1,199 tensors, including 333 vision tensors and 15 source MTP tensors

Validated component status

• Text generation: passed
• Tool calling: passed all five native-format cases
• Vision and video: passed all three deterministic local-image cases
• Thinking controls and chat template: preserved and checked against the locked source
• Hybrid Gated DeltaNet/full-attention structure: retained
• MTP tensors and metadata: retained; speculative acceleration is not advertised
• Per-file runtime smoke: passed at 11.979634621144056 generated tokens/s on the validation host
• Weight SHA-256: 156933a6436a9e9ceac2f61385993a788168ccc18760496bdc9a372d0fdf2bd2
• Validation lineage: the same bytes passed in the combined release at f519a212d6c15cd3292b6ca835dd8ebf235642c0

validation_result.json records the exact per-file smoke result and its inherited aggregate functional gates. These split repositories do not claim a new benchmark run; they preserve byte-identical, already-validated artifacts.

Inference

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout 5f754ea0e2fd21e1213db7ebebfd65d938d9d69c
cmake -S . -B build -DGGML_METAL=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

hf download Chungulus/Qwen3.8-27B-Q5_K_M-GGUF --local-dir ../qwen38-q5_k_m
cd ../qwen38-q5_k_m
../llama.cpp/build/bin/llama-mtmd-cli \
  -m ./Qwen3.8-27B-Q5_K_M.gguf \
  --mmproj ./mmproj-Qwen3.8-27B-F16.gguf \
  -p 'Describe the image.' --image ./image.png

Allow at least approximately 24 GB of available memory for the model, vision projector, and modest runtime overhead; KV-cache needs grow with context. This is planning guidance, not a measured peak-memory guarantee.

Use the source chat-template controls for enable_thinking, reasoning_effort, and preserve_thinking, together with Qwen's native tool-call format.

Limitations

Quantization can reduce quality, particularly at lower bit widths. The longest recorded validation prompt was 73 tokens; do not infer that the architectural maximum context was tested. A runtime must support the Qwen3.8 hybrid language graph, vision tower/projector, tokenizer, and retained MTP metadata rather than loading only a single language tensor.

License and attribution

The parent model and this vanilla quantization use the source model's Apache-2.0 license. See Qwen/Qwen3.8-27B.