GGUF quantizations of a custom Qwen3.8-27B model.
| Quantization | Size | File |
|---|---|---|
| Q8_0 | 28.60 GB | Qwen3.8-27B-heretic-NSFW-Q8_0.gguf |
| Q4_K_M | 16.22 GB | Qwen3.8-27B-heretic-NSFW-Q4_K_M.gguf |
| Q3_K_M | 12.96 GB | Qwen3.8-27B-heretic-NSFW-Q3_K_M.gguf |
| Q2_K | 10.12 GB | Qwen3.8-27B-heretic-NSFW-Q2_K.gguf |
| BF16 mmproj | 0.93 GB | mmproj-Qwen3.8-27B-heretic-NSFW-BF16.gguf |
Each text model is quantized independently from a BF16 GGUF intermediate using llama.cpp. The tokenizer and chat template are embedded in each text GGUF.
For image input, load the BF16 mmproj together with any one of the four text models. The same mmproj is shared by all four quantizations. The image encoder/projector uses BF16, with F32 tensors retained where required by the converter.
yukiworks/Qwen3.8-27B-heretic-NSFW