Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-GGUF

A family of six GGUF files of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, DavidAU's "Cold Fusion" (GAIN + Unsloth) tune of Qwen3.8-27B: a 27B dense hybrid model (Gated DeltaNet + Gated Attention every fourth layer, 262K native context, embedded MTP speculative head, native vision tower). The conversion source is windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4, their NVFP4 quantization of DavidAU's original. The MTP head is baked into every file; no separate drafter is needed (--spec-type draft-mtp).

My part here is only the numerics: I converted the NVFP4 checkpoint to GGUF and built a size/precision ladder for the tensors that most affect output quality and decode speed. All credit for the model itself belongs upstream (full chain below).

Follow along & support

I post updates on new conversions, benchmarks, and what I'm working on over on my site. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, you will find the options there. I do this on consumer hardware and like seeing how far it goes. More is on the way.

🌐 esatapedico.com. Updates, work-in-progress, and ways to support.

The six files

Every tier carries the same byte-identical 448-tensor native NVFP4 backbone (per-tensor SHA-256 verified) and differs only in the precision of the LM head, token embedding, and MTP draft head:

Picking a tier: MID-HIGH is the highest-precision compact option (all three head groups at Q8_0) and our fastest compact decode on dual-GPU split; LOW/VERY-LOW trade some head precision for ~2 GB less VRAM; HIGH/VERY-HIGH restore BF16 heads where VRAM allows.

Tensor layout

The source NVFP4 checkpoint (windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4) quantizes all Linear layers to true NVFP4 (group size 16, fp8-e4m3 scales) and deliberately keeps the vision tower, the Gated DeltaNet linear-attention path, lm_head, embeddings, and the MTP head in BF16.

ORIG parent (kept locally, 27.65 GB, not published): 1,122 tensors = 256 NVFP4 (MLP everywhere + attention projections on the 16 full-attention layers) + 240 BF16 (DeltaNet path) + BF16 heads/MTP/vision-side metadata + F32 norms/scales.

Compact tiers (published, 6 files): built from ORIG with llama-quantize --tensor-type-file, normalizing the whole language-model backbone to a uniform 448-tensor NVFP4 block (the tiny ssm_alpha vectors stay F32, matching our other Qwen3.8 families). The 256 source-NVFP4 tensors pass through untouched (byte-identical, SHA-256 verified against the converted parent); the DeltaNet path is re-typed from BF16 to NVFP4 once during tier building and is then frozen byte-identical across all tiers.

Vision

The Cold Fusion tune left the original Qwen3.8 vision tower untouched: we compared DavidAU's published mmproj-BF16.gguf tensor-by-tensor against the base model's projector and all 334 tensors are byte-identical. Pair any tier with his mmproj-BF16.gguf via --mmproj.

How this was made

1. Converted windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4 (compressed-tensors NVFP4A16) to an ORIG parent GGUF with convert_hf_to_gguf.py --outtype auto.
2. Generated per-tier tensor-type override maps (included below) from the parent inventory.
3. Built each tier with llama-quantize --tensor-type-file <overrides> over the shared backbone.
4. Verified: per-tier check_nvfp4 gate (448 NVFP4 tensors each), per-tensor SHA-256 backbone identity across all tiers, and passthrough integrity of the 256 source-NVFP4 tensors.

Rough sanity observations, not a benchmark (single runs, dual RTX 5070 Ti / 5060 Ti 16 GB, tensor split, llama-bench pp512/tg128):

Perplexity stays flat within ~2.6% across the whole ladder, so the quantization costs almost nothing even at the smallest tier; decode speed tracks head precision (smaller heads decode faster).

Serving check through LocalAI/llama.cpp on the same dual-16 GB box (fresh process per tier, 180k-token payload, single run each; decode samples are short because every tier stopped naturally, so treat decode as indicative only). This is a shared inference box, so runs can collide with other tenants:

Prefill is flat across the ladder (~640-650 tok/s at 180k context) and the embedded MTP head speculates at 0.80-0.93 acceptance everywhere.

SHA-256

bc67b5c698547ea38beaca6847b40b3952c8bc0891c7945fabdf30102b5cef60  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-VERY-LOW.gguf
c1aca6c2310d477e1cc053620dd1f1432833675a9a4785fac187ed26254244c1  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-LOW.gguf
2346b7639d6d66f61009e6b715b80e84d1088bbb8393b4e12a507a6fb4c9f0d4  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MEDIUM.gguf
77add63c9c551a00f7b74b1d6b347c34b122be420b38e61a73243c5e88e5254b  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf
a32a2d161ebdfeb2178bcd5e610ec5db548aba420b6b4ace3c854633a3600262  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-HIGH.gguf
af68ca2711681076ddcd3087d0117f5cecb6a30192ff614f10cc353456f0a555  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-VERY-HIGH.gguf

Attribution & provenance

This is a derivative work built entirely from existing Apache-2.0 artifacts. Nothing here was trained or fine-tuned. Credit belongs to:

1. Alibaba / Qwen team for the base model, Qwen/Qwen3.8-27B (Apache-2.0): 27B dense, 64 blocks, Gated DeltaNet + Gated Attention hybrid, native vision-language, 262,144-token context, MTP head.
2. DavidAU for the tune itself, Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 (Apache-2.0): the "Cold Fusion" (GAIN + Unsloth) training that raises general intelligence while cutting thinking tokens to a fraction of stock, and for publishing the GGUFs and discussions this work builds on.
3. Unsloth, whose trainers/systems power the Cold Fusion method.
4. windowsxp811203 for the NVFP4 checkpoint this family converts (...-NVFP4, Apache-2.0).
5. This repo's author for the GGUF conversion and the tier ladder only.

Repository contents

• Six tier GGUFs (table above)
• overrides-{very-low,low,medium,mid-high,high,very-high}.txt: per-tensor quantization maps (1,122 entries each) for reproduction

Card written with AI assistance.