Sabi-Noema GGUF Quantizations
See the original model card for details (description, benchmarks, generation parameters).
Speculative decoding
sabi didn't include spec decoding tensors, but you can try one from any other Qwen3.8-27B GGUFs. For example:
- ggml-org/Qwen3.8-27B-GGUF — MTP and DFLASH
- unsloth/Qwen3.8-27B-GGUF — MTP only
I'm getting around 40%+ acc rate with Unsloth Q4_0 MTP.
Usage
llama-cli -m Sabi-Noema-Q4_K_M.gguf --mmproj mmproj-Sabi-Noema-BF16.gguf --model-draft mtp-Qwen3.8-27B-Q4_0.gguf --spec-type draft-mtp
Conversion
llama.cpp 23deec025 (v0.6.0)