Qwen3-8B — GGUF Quants
Quantized GGUF versions of Qwen/Qwen3-8B — Alibaba's flagship Qwen3 base model at 8B parameters. Qwen3-8B delivers strong performance across reasoning, coding, and multilingual tasks, a significant leap over Qwen2.5-7B.
Available Files
| File | Quant | Size | Use Case |
|---|---|---|---|
Qwen3-8B-Q8_0.gguf |
Q8_0 | ~8.6GB | Maximum quality |
Qwen3-8B-Q6_K.gguf |
Q6_K | ~6.6GB | Near-lossless |
Qwen3-8B-Q5_K_M.gguf |
Q5_K_M | ~5.7GB | High quality |
Qwen3-8B-Q4_K_M.gguf |
Q4_K_M | ~4.9GB | Recommended default |
Qwen3-8B-Q3_K_M.gguf |
Q3_K_M | ~3.9GB | Low VRAM |
Qwen3-8B-IQ4_XS.gguf |
IQ4_XS | ~4.3GB | Imatrix 4-bit |
Qwen3-8B-IQ3_XXS.gguf |
IQ3_XXS | ~3.2GB | Imatrix 3-bit |
Qwen3-8B-IQ2_M.gguf |
IQ2_M | ~2.8GB | Imatrix 2-bit |
Qwen3-8B-IQ1_S.gguf |
IQ1_S | ~2.0GB | Extreme compression |
Qwen3-8B-fp16.gguf |
FP16 | ~16.0GB | Full precision |
imatrix.dat |
— | — | Importance matrix |
Usage
./llama-cli -m Qwen3-8B-Q4_K_M.gguf --ctx-size 8192 -n 512 \
-p "<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n"
- Parameters: 8B | License: Apache 2.0 | Context: 32K tokens
Quantized by DuoNeural using llama.cpp on RTX 5090.
DuoNeural
DuoNeural is an open AI research lab — human + AI in collaboration.
| Platform | Link |
|---|---|
| HuggingFace | huggingface.co/DuoNeural |
| Website | duoneural.com |
| GitHub | github.com/DuoNeural |
| X / Twitter | @DuoNeural |
| duoneural@proton.me | |
| Newsletter | duoneural.beehiiv.com |
| Support | buymeacoffee.com/duoneural |
DuoNeural Research Publications
Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.