Wicked-Tutu-31B NVFP4 (ComfyUI)

ReadyArt/Wicked-Tutu-31B-Final, a Gemma 4 31B roleplay finetune, quantized to NVFP4 with AWQ for ComfyUI's Generate Text node. 62.5 GB of bf16 weights become one 20.5 GB file that fits a 32 GB GPU.

ComfyUI only. This is ComfyUI's text-encoder format, not a transformers or llama.cpp checkpoint. For LM Studio / llama.cpp use ReadyArt's GGUFs.

Use

1. Patch ComfyUI first. As of October 2026 ComfyUI's Gemma 4 31B config is missing two settings, so any Gemma 4 31B checkpoint fails to generate (Missing weight for layer model.layers.5.self_attn.v_proj at load, then shape '[1, N, 8192]' is invalid). In comfy/text_encoders/gemma4.py, add to class Gemma4_31B_Config:

   num_global_key_value_heads = 4
   attention_k_eq_v = True
   

   These match google/gemma-4-31B-it's config. Restart ComfyUI. Upstream fix: Comfy-Org/ComfyUI#17220; once it is merged, this step is no longer needed.

2. Put the file in models/text_encoders/.

3. Load CLIP → Generate Text. The type can stay on its default; Gemma 4 31B is detected from the weights. Keep use default template on, or the model receives raw text instead of chat turns and degenerates.

vision_model…input_max / input_min / output_max / output_min "missing" warnings at load are expected: the 31B vision tower has no clipped linears, and ComfyUI's default (no clipping) is the correct behaviour. Image input is untested.

On an RTX 5090: ~21.3 GB peak VRAM, ~12.5 tokens/s greedy.

Quality

Greedy outputs on the same prompts (prompt rewrite, a roleplay opener, a short factual answer), compared with the bf16 model:

That is three prompts, not a benchmark — try it on your own prompts.

How it was made

• Recipe: NVIDIA ModelOpt NVFP4_AWQ_LITE_CFG: NVFP4 weights (FP4 E2M1 values, FP8 scale per 16 weights, one FP32 scale per tensor) with AWQ per-channel smoothing. All 410 language-model linears (q/k/v/o, gate/up/down) are quantized. Embeddings (tied with the output head), norms and the vision tower stay bf16.
• Calibration (~207k tokens, rendered with the model's chat template, thinking off):
  • 64 multi-turn chats from UltraChat 200k
  • 64 character roleplays (card as system prompt) from PIPPA
  • 64 short image-prompt ideas from Stable-Diffusion-Prompts, answered by the bf16 model itself, so calibration sees the model's own rewrites
• Export: ComfyUI's mixed-precision format, packed directly from the calibrated quantizers (no re-quantization): each linear carries weight (packed FP4), weight_scale, weight_scale_2, input_scale, the AWQ pre_quant_scale (which ComfyUI applies to the input) and comfy_quant. The tokenizer is embedded as tokenizer_json.
• Runtime note: ComfyUI runs text encoders with 4-bit weights and bf16 activations, so the activation scales are calibrated but unused today. AWQ was measured to help in the stricter W4A4 setting too (next-token KL to bf16 1.08 vs 1.28 nats for plain NVFP4, top-1 agreement 63.7% vs 59.9%, on held-out chat / roleplay / rewrite text). Those W4A4 numbers describe 4-bit activations, not what ComfyUI runs.

Licence

Apache 2.0, as Gemma 4 and the finetune. ReadyArt's terms are included unchanged in LICENSE.txt: 18+ only, you take full responsibility for generated content, personal use only.

Credits: ReadyArt for Wicked-Tutu; Google for Gemma 4; NVIDIA ModelOpt.