Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-GGUF

HF repos: esatapedico/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4 (safetensors, compressed-tensors NVFP4) and esatapedico/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-GGUF (this GGUF family, 6 tiers).

A family of six GGUF files of DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored, DavidAU's TWIN-TURBO tune (Fable plus Cold Fusion plus Heretic/Uncensored on Qwen3.8-27B, tuned for sharply reduced thinking tokens at matched quality): a 27B dense hybrid model (Gated DeltaNet plus Gated Attention every fourth layer, 262K native context, embedded MTP speculative head, native vision tower). NVFP4 here means W4A16 with FP8 scales, group size 16, weight only. Linear layers are NVFP4, vision tower, linear attention path, lm_head, embeddings and MTP head are kept in BF16 at the quantization source. The MTP head is baked into every file in this repo, no separate drafter is needed (--spec-type draft-mtp).

My part here is only the numerics: I converted the NVFP4 checkpoint to GGUF and built a size and precision ladder for the tensors that most affect output quality and decode speed. All credit for the model itself belongs upstream (full chain below).

Update 2026-09-15: multi-modal template fix

All six files were re-issued with the upstream multi-modal fix from the same tool-call thread (the REASON-scanner pass no longer flags every message as system content, so image and video messages render vision tokens instead of raising; the system-message image guard is retained). Keeps the 2026-09-12 toolcall2 fixes plus the string-argument guard (upstream's reg file still lacks the latter, so our template stays ahead on that point; credit apollo-mg, donboyle, DavidAU). Tensor data is byte-identical to the previous release; only the embedded template changed, so each file grew by 544 bytes and every SHA-256 below is new.

Update 2026-09-12: tool-call template fix

All six files were re-issued with the corrected chat template from the upstream tool-call thread (preserved tool_calls, REASON-scanner guards, string-argument guard; credit apollo-mg, donboyle, DavidAU). Tensor data is byte-identical to the previous release; only the embedded template changed, so each file grew by 544 bytes and every SHA-256 below is new. For tool calling, upstream recommends reasoning mode with temp 0.6 to 0.7; instruct-mode tool calling stays poor.

Follow along & support

I post updates on new conversions, benchmarks, and what I'm working on over on Ko-fi. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, a coffee is always welcome. I do this on consumer hardware and like seeing how far it goes. More is on the way.

☕ ko-fi.com/esatapedico. Updates, work-in-progress, and an optional coffee.

The six files

Six tiers in this repo share the MTP head (block_count 65, nextn_predict_layers 1, 1,122 tensors) and a byte-identical 448-tensor native NVFP4 backbone, differing only in lm_head / token_embd / MTP precision:

Sizes are decimal GB as shown in the HF file browser. Picking a tier: MID-HIGH is the highest precision compact option (all three head groups at Q8_0); LOW trades some head precision for about 1.4 GB less VRAM; HIGH and VERY-HIGH restore BF16 heads where VRAM allows. VERY-HIGH needs about 18 GiB VRAM plus KV, so single 16 GB cards will fall back to CPU offload.

Tensor layout

At a high level, the GGUFs in this repo contain 1,122 tensors with a uniform 448-tensor native NVFP4 block shared byte identical across all six tiers. The NVFP4 format is W4A16 with group size 16 and FP8 E4M3 scales. Remaining tensors are F32 norms and scales, with lm_head, token embeddings and MTP head varying per tier as in the table above.

Vision

The TWIN-TURBO tune leaves the original Qwen3.8 vision tower in place. Pair any tier with the matching vision projector mmproj-BF16.gguf via --mmproj. No mmproj is bundled in this repo, use the projector from the base model.

Agentic use and long tasks

Two community-reported behaviors matter if you drive this model with tools or agents. First, INSTRUCT mode (enable_thinking:false) does not reliably chain tool calls on this tune: one measured series managed 1 chained second call in 12 tries, and the failure persisted across templates and quants. For agentic loops, keep thinking on with a low reasoning effort instead. Second, if generations stop early in chat apps, set explicit stop tokens (<|im_start|>, <|im_end|>, <|endoftext|>) and keep sampling plain (temperature 0.6 coding / 0.7 general, top_k 20, top_p 0.95, repeat_penalty 1.0, presence penalty off). Users who prefer a different chat template can pass --chat-template-file with any llama.cpp run; the file embedded here is DavidAU's own template, unchanged.

How this was made

At a high level, the steps were:

1. Converted the BF16 base model to compressed tensors NVFP4 (W4A16, vision, linear attention, lm_head and MTP kept in BF16, no calibration).
2. Verified NVFP4 metadata and restored the 15 MTP tensors as BF16.
3. Converted the NVFP4 checkpoint to GGUF.
4. Built each tier over a shared 448 tensor NVFP4 backbone, varying only lm_head, token embedding and MTP head precision per tier.
5. Verified per tier NVFP4 tensor count and backbone byte identity across all tiers, and patched GGUF KV for name, description and license without changing tensor data.

Benchmarks (naive, single run, not comparable across setups)

These are rough sanity checks to confirm the files load and generate, not a formal benchmark. Method and hardware are noted so you can interpret them in context. Results will vary with hardware, sampling and context length.

All runs used direct GPU via llama.cpp tools, with a diverse synthetic English payload (about 75 kB). Gateway numbers below are end to end through an OpenAI-compatible serving endpoint and include serving overhead, don't compare them to llama-bench decode. Gateway rows beyond VERY-LOW are still being measured and will be added here as they land.

Smoke: 6 of 6 coherent

Prompt What is a black hole? via single turn sampling, each tier produces a natural coherent completion with no repetition or truncation. All tiers passed.

Perplexity (PPL)

Direct llama-perplexity on the 75 kB diverse payload, single GPU, same chunks for all tiers:

The six tiers span only 0.046 from best to worst, so the quantization costs almost nothing even at the smallest tier.

Speed (llama-bench, pp512 prompt processing, tg128 generation, tok/s)

Single 5070 Ti (16 GB class) with full CUDA offload, three runs per tier:

Gateway check (naive, one-at-a-time)

Each tier gets one large request with max_tokens=20000. These are end to end timings through the serving endpoint, not pure decode, so treat as sanity and responsiveness, not a formal benchmark.

SHA-256

f842d96be7411079b8933af8529d86e5fae9de851084be112b7b533c7b77d85d  Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-VERY-LOW.gguf
2733f98901c7eaeae551e3894d1eafbd5f3cbcb921959ea8aaa774396abd4c58  Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-LOW.gguf
34527d55224cd0e2ebbcc4fdf8b11a1eb73e975d8735e1e2fc6e87040b4fc1c3  Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-MEDIUM.gguf
ed2d2e482d770b682376dba28c63e9711f61aa1a77a1a5d48c1eaa8a7d6f9063  Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-MID-HIGH.gguf
f580b06aad00fe3bb743d1570473e28765f0a59bc536faa90d45b21f87cca7ae  Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-HIGH.gguf
f3b524d777fe724182ff3bd9619cdf1a9a17cc319d4427995de2508915b7e054  Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4-VERY-HIGH.gguf

Attribution & provenance

This is a derivative work built entirely from existing Apache 2.0 artifacts. Nothing here was trained or fine tuned. Credit belongs to:

1. Alibaba and Qwen team for the base model, Qwen/Qwen3.8-27B (Apache 2.0): 27B dense, 64 blocks, Gated DeltaNet plus Gated Attention hybrid, native vision language, 262,144 token context, MTP head.
2. DavidAU for the tune itself, Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored (Apache 2.0): the TWIN-TURBO Fable plus Cold Fusion plus Heretic/Uncensored stack that this family converts.
3. Unsloth, whose trainers and systems power the underlying training methods.
4. This repo's author for the GGUF conversion and the tier ladder only.

Repository contents

• Six tier GGUFs (table above, 14.86 to 19.69 GB) plus 6 override maps (overrides-*.txt, 1,122 lines each: per-tensor target types, includes output.weight/token_embd.weight per tier).
• Corresponding safetensors NVFP4 checkpoint (single model.safetensors, compressed tensors nvfp4-pack-quantized): esatapedico/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-L-Uncensored-NVFP4

License

apache-2.0 (inherits from Qwen base and DavidAU tune). general.license = apache-2.0 is set inside every GGUF. general.name matches the tier name, qwen35.block_count 65, qwen35.context_length 262144.

Card written by AI assistance at the request of the repository author, who did the engineering. As with any AI-generated text, there may be errors; please verify anything important (hashes, sizes, commands) against the file itself before relying on it.