Qwen3.5-0.8B-Opus-Reasoning-Distilled — GGUF

GGUF quantizations of PhilipEriksson/Qwen3.5-0.8B-Opus-Reasoning-Distilled, a 0.8B reasoning model fine-tuned on Claude Opus and Qwen3.5-27B reasoning traces.

Files

File	Quant	Size	Bits/weight
`qwopus-0.8b-Q8_0.gguf`	Q8_0	834 MB	8.5
`qwopus-0.8b-Q4_K_M.gguf`	Q4_K_M	542 MB	4.9

Both are single-file, self-contained, and include the tokenizer.

Usage

llama.cpp:

llama-cli -m qwopus-0.8b-Q4_K_M.gguf \
  -p "<|im_start|>user\nExplain why the sky is blue.<|im_end|>\n<|im_start|>assistant\n" \
  -n 512 --temp 0.6

Ollama:

ollama run hf.co/PhilipEriksson/Qwen3.5-0.8B-Opus-Reasoning-Distilled-GGUF:Q4_K_M

LM Studio: search the repo name, pick a quant.

The model emits a thinking...</think> block before its answer. llama.cpp renders the closing token as [End thinking] rather than the raw token — that is expected, not a truncated generation.

Measured quality

Three prompts (arithmetic, code, logic) run through every artifact of this model, same sampler settings, max_new_tokens=700:

Artifact	Size	Expected answer found
BF16 (baseline)	1.7 GB	3/3
GGUF Q8_0	834 MB	3/3
GGUF Q4_K_M	542 MB	3/3
EXL3 4.0 bpw	1.08 GB	3/3
EXL3 3.0 bpw	1.02 GB	2/3
NVFP4	892 MB	3/3

Both GGUF quants match the BF16 baseline on this set. Three prompts is a smoke test, not a benchmark — it catches gross breakage and obvious cliffs, it cannot rank formats by quality.

How these were produced

Converted with Unsloth's bundled llama.cpp converter, then quantized with llama-quantize. Stock llama.cpp's convert_hf_to_gguf.py is broken for the qwen3_5 architecture (issue #27019, closed as not planned): _reorder_v_heads hardcodes head_dim=1 for the linear-attention V-head reorder and does not expand in_proj_a/in_proj_b. The runtime was never affected — only the converter.

Limitations

Learning run, not a benchmark-competitive release. 2,917 training rows, 25 minutes on one RTX 5090.
Text only. The vision tower was excluded from the fine-tune; the multimodal architecture is inherited from the base model but was not trained.
0.8B parameters. Expect it to fail at anything requiring real world knowledge or multi-step planning. It is good at producing the shape of a reasoning trace.
Inherits the base model's apache-2.0 license.