Swift-Qwen3.8-27B-Genesis — NVFP4 GGUF

NVFP4 quantisation of LuffyTheFox/Swift-Qwen3.8-27B-Genesis-F16-GGUF, built with advanced-gguf-quantizer (a llama.cpp fork focused on NVFP4 quantization).

Base model: ukisai/Swift-Qwen3.8-27B-GGUF with Genesis tensor repair by LuffyTheFox.

Files

v4 — Recommended (inline scales, LM Studio compatible)

File	Calibration	MTP	Size	Tensors
`Swift-Qwen3.8-27B-Genesis-NVFP4-v4.gguf`	none (data-free)	yes (blk.64 preserved)	15.76 GB	866
`Swift-Qwen3.8-27B-Genesis-NVFP4-v4-noMTP.gguf`	none (data-free)	no (blk.64 stripped)	15.53 GB	851

v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales without separate .scale/.input_scale tensors. Required for LM Studio / Pelican and other runtimes that do not support the extended NVFP4 scale tensor contract.

Imatrix variants will follow after the plain v4 is confirmed working in target runtimes.

Source


Source GGUF	`Swift-Qwen3.8-27B-F16-0000{1..3}-of-00003.gguf` (55.6 GB total, split)
Architecture	`qwen35` (dense), 64 layers + 1 MTP/NextN block
Layers	48 Gated DeltaNet + 16 gated-attention
`general.file_type`	39 (`LLAMA_FTYPE_MOSTLY_NVFP4`)
MTP/NextN	`qwen35.nextn_predict_layers=1` (native, preserved)

Single-step F16 → NVFP4 — the F16 source gives the NVFP4 encoder clean data with no intermediate quantization noise. llama-quantize reads the split source directly via ...-00001-of-00003.gguf.

Tensor mix (v4)

type	count	notes
F32	360	norms, ssm scalars (a/dt/conv1d), nextn norms
F16	4	blk.0 attn_gate/attn_qkv, blk.0/blk.13 ffn_down
NVFP4	502	bulk weights incl. output.weight, token_embd
total	866
separate `.scale`/`.input_scale`	0	inline UE4M3 only

Tensor protection policy

F16 singular-collapse protection:

tensor	type
`blk.0.attn_gate.weight`	F16
`blk.0.attn_qkv.weight`	F16
`blk.0.ffn_down.weight`	F16
`blk.13.ffn_down.weight`	F16

F32 architecture-specific protection:

blk.*.attn_norm.weight, blk.*.post_attention_norm.weight
blk.*.attn_q_norm.weight, blk.*.attn_k_norm.weight
blk.*.ssm_norm.weight, blk.*.nextn.*.norm.weight
output_norm.weight
blk.*.ssm_conv1d.weight, blk.*.ssm_dt.bias, blk.*.ssm_a

Forced NVFP4 (do not push lower):

tensor	type
`blk.0.ssm_out.weight`	NVFP4
`blk.1.attn_gate.weight`	NVFP4
`blk.1.attn_qkv.weight`	NVFP4

Usage

llama-cli -m Swift-Qwen3.8-27B-Genesis-NVFP4-v4.gguf \
  --mmproj mmproj-Swift-Qwen3.8-27B-F16.gguf \
  --jinja -c 131072 -ngl 99

For the MTP variant, use Swift-Qwen3.8-27B-Genesis-NVFP4-v4-noMTP.gguf when the runtime lacks FastMTP support, or the MTP file for speculative decoding.

Set K cache and V cache quantization to F16
Vision support requires the mmproj-Swift-Qwen3.8-27B-F16.gguf from the source repository

Hardware

Blackwell (RTX 50xx): native FP4 path, fastest
Ampere (RTX 30xx): NVFP4 inference works via fallback kernels
Quantisation was done CPU-only (Ampere CUDA NVFP4 encoder is unreliable)

Reproducibility

# F16 split source -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
  --allow-requantize --mode fast --nvfp4-inline-scales-only \
  --tensor-type '.*=nvfp4' \
  --tensor-type '^blk\..*\.attn_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.post_attention_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.attn_q_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.attn_k_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.nextn\..*norm\.weight$=f32' \
  --tensor-type '^output_norm\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_conv1d\.weight$=f32' \
  --tensor-type '^blk\..*\.ssm_dt\.bias$=f32' \
  --tensor-type '^blk\..*\.ssm_a$=f32' \
  --tensor-type '^blk.0.ssm_out.weight$=nvfp4' \
  --tensor-type '^blk.1.attn_gate.weight$=nvfp4' \
  --tensor-type '^blk.1.attn_qkv.weight$=nvfp4' \
  --tensor-type '^blk.0.attn_gate.weight$=f16' \
  --tensor-type '^blk.0.attn_qkv.weight$=f16' \
  --tensor-type '^blk.0.ffn_down.weight$=f16' \
  --tensor-type '^blk.13.ffn_down.weight$=f16' \
  Swift-Qwen3.8-27B-F16-00001-of-00003.gguf \
  Swift-Qwen3.8-27B-Genesis-NVFP4-v4.gguf NVFP4 6

Credits

Base model: ukisai/Swift-Qwen3.8-27B
Genesis tensor repair + F16 GGUF: LuffyTheFox
NVFP4 quantisation: jan1k
Quantiser: advanced-gguf-quantizer

jan1k/Swift-Qwen3.8-27B-Genesis-NVFP4-GGUF