Qwen3.8-27B GGUF Model Hub
Unified release repository for Qwen3.8-27B quantizations, fine-tunes, and speculative decoding configurations optimized for llamAmpere on SM86 (RTX 3090 / 3090 Ti).
Available Models
| File | Base Architecture | BPW / Quant | MTP Drafter | Target Hardware / Fit |
|---|---|---|---|---|
qwen3.8-27b-ternary-mtp.gguf |
Prism ML Ternary Bonsai 2 27B | 1.75 bpw (PTQ1_0) |
Native 1-layer MTP | 5.81 GiB; fits 262K context under 24 GB VRAM |
Qwen3.8-27B-ATX-4-XS.gguf |
Qwen3.8-27B Base | 4.56 bpw (ATX-IQ4_XS-M) |
Native 1-layer MTP | 14.52 GiB; 245K context populated under 22.6 GB ready |
ATX-Swift-Qwen3.8-27B-Uncensored-IQ4_XS-M.gguf |
Qwen3.8-27B Swift Uncensored | 4.56 bpw (IQ4_XS-M) |
Native 1-layer MTP | 15.58 GiB |
Qwopus3.8-27B-Flash-ATX-IQ4_XS-M.gguf |
Qwopus3.8-27B Flash | 4.56 bpw (IQ4_XS-M) |
Native 1-layer MTP | 15.58 GiB |
Running with llamAmpere
Build the SM86 runtime:
cmake -S . -B build-sm86 -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DGGML_CUDA_FA=ON \
-DCMAKE_CUDA_ARCHITECTURES=86 -DGGML_NATIVE=ON
cmake --build build-sm86 -j8 --target llama-server
Run with native MTP speculative drafting:
./build-sm86/bin/llama-server \
-m qwen3.8-27b-ternary-mtp.gguf \
-c 32768 -b 4096 -ub 1024 -t 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
--spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0