Qwen3.6-35B-A3B-Uncensored-Genesis — NVFP4 GGUF

NVFP4 quantisation of LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Genesis-Final-GGUF, built with advanced-gguf-quantizer (a llama.cpp fork focused on NVFP4/MXFP6 quantization).

This is the non-Hermes version — Genesis tensor repair on the HauhauCS uncensored base, without the Hermes finetune transfer. For the Hermes version, see jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-Hermes-Final-NVFP4-GGUF.

Files

v4 — Recommended (inline scales, LM Studio compatible)

File	Size	Tensors
`Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf`	~20 GB	733

v4 uses --nvfp4-inline-scales-only to emit native inline UE4M3 scales without separate .scale/.input_scale tensors. This is required for LM Studio / Pelican and other runtimes that do not support the extended NVFP4 scale tensor contract.

Imatrix variants will follow after the plain v4 is confirmed working in target runtimes.

Source


Source GGUF	`Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P.gguf` (43.6 GB, 10.06 BPW)
Architecture	`qwen35moe` (MoE), 35B total / ~3B active, 40 layers
Context	262144
`general.file_type`	39 (`LLAMA_FTYPE_MOSTLY_NVFP4`)
MTP/NextN	none

Two-step pipeline: Q8_K_P -> F16 intermediate -> NVFP4. The F16 step gives the NVFP4 encoder clean data (no Q8_0 quantization noise).

Tensor mix (v4)

type	count	notes
F32	331	norms, ssm scalars, gate inputs
F16	175	sensitive weights (blk.0 attn, ssm, etc.)
Q6_K	1	`output.weight`
NVFP4	226	bulk weights
total	733
separate `.scale`/`.input_scale`	0	inline UE4M3 only

Format fix (v3/v2 -> v4)

The older v2/v3 files use the extended NVFP4 format with separate .scale and .input_scale tensors. Runtimes like LM Studio / Pelican only read inline UE4M3 scales and ignore the separate scale tensors, producing garbage output.

v4 uses --nvfp4-inline-scales-only to:

Skip writing .scale and .input_scale tensors
Force tensor_scale=1.0 (no pre-scaling)
Force input_scale=identity
Make inline UE4M3 scales self-contained

Tensor protection policy

F16 singular-collapse protection:

tensor	type
`blk.0.attn_gate.weight`	F16
`blk.0.attn_qkv.weight`	F16
`blk.0.ffn_down_exps.weight`	F16
`blk.13.ffn_down_exps.weight`	F16

F32 architecture-specific protection:

tensor	reason
`blk.*.attn_norm.weight`	1D norm
`blk.*.post_attention_norm.weight`	1D norm
`blk.*.attn_q_norm.weight`	1D norm
`blk.*.attn_k_norm.weight`	1D norm
`blk.*.ssm_norm.weight`	1D norm
`output_norm.weight`	1D norm
`blk.*.ssm_conv1d.weight`	CUDA SSM conv kernel requires F32
`blk.*.ssm_dt.bias`	SSM scalar, kernel compatibility
`blk.*.ssm_a`	SSM scalar, kernel compatibility
`blk.*.ffn_gate_inp_shexp.weight`	shared expert gate input
`token_embd.weight`	F16 (embedding quality)

Forced NVFP4 (do not push lower):

tensor	type
`blk.0.ssm_out.weight`	NVFP4
`blk.1.attn_gate.weight`	NVFP4
`blk.1.attn_qkv.weight`	NVFP4

Usage

llama-cli -m Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Uncensored-Genesis-F16.gguf \
  --jinja -c 131072 -ngl 99

Set K cache and V cache quantization to F16
Set GPU offload to maximum, active experts to 8
Set number of layers for which to force MoE weights onto CPU to 40

Hardware

Blackwell (RTX 50xx): native FP4 path, fastest
Ampere (RTX 30xx): NVFP4 inference works via fallback kernels
Quantisation was done CPU-only (Ampere CUDA NVFP4 encoder hangs on MoE)

Reproducibility

# Step 1: Q8_K_P -> F16 intermediate
llama-quantize --allow-requantize \
  Qwen3.6-35B-A3B-Uncensored-Genesis-Q8_K_P.gguf temp_f16.gguf F16 6

# Step 2: F16 -> NVFP4 v4 (inline scales only, CPU-only)
CUDA_VISIBLE_DEVICES="" llama-quantize \
  --allow-requantize \
  --nvfp4-inline-scales-only \
  --tensor-type-file tensor_types_protection.txt \
  temp_f16.gguf \
  Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-v4.gguf \
  NVFP4 6

# Cleanup
rm temp_f16.gguf

Credits

Base model: HauhauCS (0/465 refusals)
Genesis tensor repair: LuffyTheFox
NVFP4 quantisation: jan1k
Quantiser: advanced-gguf-quantizer

jan1k/Qwen3.6-35B-A3B-Uncensored-Genesis-NVFP4-GGUF