Qwen3.8-27B-Uncensored · FIT-GGUF
Fourteen continuous-size GGUF quantizations — every file exactly its predicted size, every tier measured.
7 GiBask for any budget in between13.5 GiBEnglish · 简体中文 📖
🧭 About FIT-GGUF — the tool behind these filesEvery file in this repository was planned, executed and verified by FIT-GGUF, an open-source, deterministic tensor-level planning layer on top of llama.cpp quantization presets. Standard GGUF quantization asks you to pick one of a handful of presets; FIT-GGUF starts from the largest supported preset below your requested byte budget, then spends the remaining bytes on deterministic tensor-level precision upgrades.
Traditional GGUF gives you presets. FIT gives you a size slider.
| Deterministic size prediction & recipe execution | ✅ Validated |
| Universally optimal tensor allocation | ⚠️ Not established — FIT claims precise size control, not a universal quality optimum |
The method, the full preregistered research record and the fit CLI are open source: github.com/Scorp1o117/FIT-GGUF
The source model is an abliterated, refusal-removed model with no meaningful built-in guardrails, and may comply with harmful, illegal or unsafe requests. Use it only where you can provide appropriate moderation, access control and legal review. Do not deploy it to end users without your own safety layer.
源模型经过拒答方向移除，不具备可靠的内置安全护栏；请仅在合法、受控、具备审核与滥用防护的环境中使用，使用者自行承担部署责任。
📦 Pick a tierFIT-12G means a 12 GiB budget for the main GGUF file — not total RAM/VRAM usage. KV cache, compute buffers, runtime overhead and the multimodal projector are separate. G = GiB (2³⁰ bytes). Naming: Qwen3.8-27B-Uncensored-FIT-<tier>-<dominant>.gguf
MTP removed: every quantization in this repository ships without the NextN/MTP head (the source was converted with --no-nextn). MTP-based speculative decoding is therefore not available with these files; text and vision inference are unaffected.
| Tier | GiB | Dominant | Macro KL ↓ | Same-top ↑ | Measured positioning |
|---|---|---|---|---|---|
| FIT-7G | 7.000 | IQ1_M | 1.1327 | 63.9% | Extreme compression; large measured quality loss |
| FIT-7.5G | 7.500 | IQ2_XXS | 0.5898 | 73.1% | First major quality step above the IQ1 region |
| FIT-8G | 7.999 | IQ2_XXS | 0.4838 | 76.9% | Beats the IQ2_XXS preset (0.5403) with +0.15 GiB |
| FIT-8.5G | 8.499 | IQ2_XXS | 0.4527 | 78.4% | Best measured point in the 8–9.3 GiB native region |
| FIT-9G | 8.999 | Q2_K | 0.3363 | 80.8% | q2_k-directed fill; large KL step over FIT-8.5G |
| FIT-9.5G | 9.500 | Q2_K | 0.2737 | 83.1% | Beats the Q2_K_S preset (0.2889) at slightly less size |
| FIT-10G | 9.999 | Q2_K | 0.2299 | 84.6% | Beats the Q2_K preset (0.2439); compact general tier |
| FIT-10.5G | 10.499 | IQ3_XXS | 0.1873 | 87.4% | Clear fidelity step over the 10G tier |
| FIT-11G | 10.988 | IQ3_S | 0.1515 | 88.9% | Near IQ3_XS macro quality, slightly smaller |
| FIT-11.5G | 11.434 | IQ3_S | 0.1439 | 89.2% | Near IQ3_S; documented 67.5 MiB target slack |
| ⭐ FIT-12G | 12.000 | IQ3_S | 0.1227 | 90.3% | Strongest measured quality/size point in this release |
| FIT-12.5G | 12.497 | IQ3_S | 0.1116 | 91.0% | 0.5 GiB over FIT-12G buys a clear KL step |
| FIT-13G | 12.998 | IQ4_XS | 0.0987 | 91.9% | IQ4_XS becomes the dominant payload |
| FIT-13.5G | 13.498 | IQ4_XS | 0.0838 | 92.7% | Best measured FIT-tier macro KL in this batch |
Quick picks
🏆 FIT-12G — the sweet spotBelow the IQ3_S / IQ3_M presets (0.1227 vs 0.1424 / 0.1445), far cheaper than IQ4_XS.🎯 FIT-13.5G — max qualityHighest measured quality in this batch: KL 0.0838, Same-top 92.7%.💸 FIT-8.5G — budget winnerThe 8–10 GiB region winner; beats the IQ2_XXS preset outright.⚠️ Below ~7.5 GiBQuality drops sharply — the IQ1 region is rough and reported as measured.All numbers are protocol-scoped observations (five fixed 64 KiB domains vs aligned BF16), not an application benchmark or a universal ranking.
📈 Measured qualityMeasured quality improves monotonically across all 14 tiers (macro KL 1.1327 → … → 0.0838), and in the 8–10 GiB region every FIT tier beats its surrounding llama.cpp default presets: FIT-8G / FIT-8.5G beat IQ2_XXS, FIT-9.5G beats Q2_K_S, FIT-10G beats Q2_K. Superseded early recipes (the P5/P6 repairs) are retained in the research record as evidence, not hidden.
Full-size KL · 中文大图 · Same-top · Allocation repair · Target utilization
🚀 Run itllama.cpp (text)
./llama-cli \ -m Qwen3.8-27B-Uncensored-FIT-12G-IQ3_S.gguf \ -ngl 99 \ -c 8192 \ -cnv
llama-server (text + vision)
./llama-server \ -m Qwen3.8-27B-Uncensored-FIT-12G-IQ3_S.gguf \ --mmproj mmproj-Qwen3.8-27B-Uncensored-BF16.gguf \ -ngl 99 -c 8192
Any llama.cpp-based runner (llama-cli, llama-server, LM Studio, KoboldCpp, Jan, …) loads these files directly. The BF16 projector pairs with any tier. Pick -ngl, context and batch for your hardware — and remember the GGUF file size alone is not a RAM/VRAM requirement calculator.
| Runtime | llama.cpp b10666 (commit 4e97ac86e) · Linux x86_64 · ROCm |
| Command shape | llama-perplexity -ngl 99 -t 16 -c 512 -b 512 --kl-divergence ... |
| Reference | Aligned BF16 logits from the converted source |
| Domains | wiki_test · wiki_valid · Chinese · code · agent_chat (five fixed 64 KiB slices, macro mean) |
Size accuracy: all 14 artifacts matched their post-oracle predicted byte sizes exactly; most use >99.98% of the requested target (FIT-11.5G: 99.427%, the 67.5 MiB reported as target slack — llama.cpp counter-based preset rules shift when manual overrides bypass parts of preset selection; the planner detects this via an override-aware dry-run oracle).
Allocator scope: the balanced v0.1b policy has positive holdout evidence on the development architecture at some budgets but did not beat matched random allocation on a second model family. This release claims deterministic target-size planning and reports measured quality for these specific artifacts — it does not claim a universally optimal allocation.
🧩 Included — and not included✅ 14 quantized main language-model GGUFs · ✅ BF16 multimodal projector mmproj-Qwen3.8-27B-Uncensored-BF16.gguf (vision, pairs with any tier) · per-tier plan, effective recipe, tensor override file and quantize record (fit-plans/) · full metrics (results/p4-results.json) · checksums (results/SHA256SUMS)
❌ The auxiliary NextN/MTP head was excluded from the source conversion (--no-nextn).
The original abliteration belongs to OrcaRouter; this repository contributes the FIT quantization plans and artifacts only.
🔍 Verify & reproducesha256sum -c results/SHA256SUMS
Evaluation slices, evaluation logs and the complete decision record live in the FIT-GGUF repository. Exact-size behavior is scoped to the recorded source metadata and pinned llama.cpp build; changing the converter, runtime, source layout or metadata requires revalidation.
📄 License & creditsApache-2.0, inherited from the base model — follow the upstream license and model-card requirements.
OrcaRouter — abliterated BF16 source weights · Qwen — the original model family · llama.cpp — GGUF tooling and runtime · Unsloth calibration dataset lineage — imatrix (1,251 chunks, reused from the same-architecture lineage). FIT-GGUF is an independent project, not affiliated with Qwen, Alibaba, OrcaRouter or llama.cpp.
⭐ FIT-GGUF on GitHub — the tool, the method, the full research recordEnglish · 简体中文