Qwen3-32B-lecore-19g-GGUF
GGUF builds of Qwen/Qwen3-32B for llama.cpp, Ollama and LM Studio. These are companions to the MLX export staccs/Qwen3-32B-lecore-19g-mlx, which has the measured perplexity numbers.
| File | Size | What it is |
|---|---|---|
Qwen3-32B-lecore-19g-GGUF-Q4_K_M.gguf |
19.76 GB | stock llama.cpp Q4_K_M |
Qwen3-32B-lecore-19g-GGUF-lecore-mix.gguf |
18.64 GB | leCore per-tensor mix: each linear layer gets the GGUF K-quant matching the bits leCore measured for it (2→Q2_K, 3→Q3_K, 4→Q4_K, 8→Q8_0); embeddings and output follow the MLX export |
The lecore-mix file is quantized from the original bf16 weights with llama.cpp K-quants. The bit width of each tensor follows leCore's measured (KL-damage) allocation, so its numbers are close to, but not identical to, the MLX export.
Some tensor widths are not multiples of 256, so llama.cpp may fall back to a compatible type for those tensors.
Note: at this budget every linear tensor was allocated 4 bits, so lecore-mix is in practice a uniform Q4_K build (with a Q6_K output layer). GSM8K regressed in the MLX export. See the MLX card for the diagnosis.
Usage
Ollama: download a file, then
printf 'FROM ./Qwen3-32B-lecore-19g-GGUF-Q4_K_M.gguf
' > Modelfile && ollama create qwen3-32b-lecore-19g-gguf -f Modelfile && ollama run qwen3-32b-lecore-19g-gguf
or pull straight from the Hub with ollama run hf.co/staccs/Qwen3-32B-lecore-19g-GGUF:Q4_K_M
LM Studio: search for staccs/Qwen3-32B-lecore-19g-GGUF in the model browser and pick a file. On a 36 GB Mac, the lecore-mix file is the one that fits.
llama.cpp
llama-cli -hf staccs/Qwen3-32B-lecore-19g-GGUF -p "Hello" -n 256
Benchmarks (lm-eval-harness 0.4.9, run on the shipped weights dequantized to bf16 on GPU)
MMLU 5-shot on the first 10 questions of each of the 57 subjects; GSM8K 5-shot on the first 100 test problems (greedy, 320 new tokens, no chat template). These are subsets, so expect about ±2–4 points of sampling noise.
| Model | MMLU (5-shot) | GSM8K strict | GSM8K flexible |
|---|---|---|---|
| leCore MLX export (numbers measured on the MLX weights; these GGUF files were not benchmarked) | 85.3% | 27.0% | 34.0% |
| stock 3-bit affine, group 64 (simulated) | 80.9% | 83.0% | 78.0% |
License
Base model: Qwen/Qwen3-32B, released by the Qwen team under the Apache License 2.0 (https://huggingface.co/Qwen/Qwen3-32B/blob/main/LICENSE). Only quantization was done here: no retraining.
Related builds
Same base model, other formats in the lecore set:
| repo | format | size |
|---|---|---|
| Qwen3-32B-lecore-19g-mlx | MLX (Apple Silicon) | 18.64 GB |
| Qwen3-32B-lecore-GGUF | GGUF (llama.cpp) | 32.88 GB |
| Qwen3-32B-lecore-hrr | gated-HRR gates | 4.4 MB |
| Qwen3-32B-lecore-mlx | MLX (Apple Silicon) | 13.01 GB |