ComfyUI Qwen3-VL GGUF (8B & 4B) — Unified Hybrid & Split `mmproj` Editions

GGUF quantizations of Qwen3-VL extracted from Comfy-Org's Qwen3-VL repackaged weights.

This repository provides two distinct formats depending on your runtime:

🔥 NEW: Unified Single-File Hybrid GGUFs (*_vision_q8_0.gguf) — Combines the K-quantized LLM backbone (Q4_K_M / Q5_K_M) and the Q8_0 Vision Tower + DeepStack Mergers + 5D Conv3d shape metadata (comfy.gguf.orig_shape) inside a single file.
Split LLM + Standalone mmproj GGUFs — Standard two-file format for llama.cpp, llama-server, Ollama, and split VLM loaders.

Model Overview

Original Weights Source: Comfy-Org/Qwen3-VL
Underlying Architecture: Alibaba Cloud Qwen3-VL (Qwen3VLForConditionalGeneration / qwen3vl)
Hybrid Format Vision Precision: Q8_0 Vision Tower & DeepStack Mergers + F32 LayerNorms/Biases + preserved 5D [1152, 3, 2, 16, 16] patch embedding metadata
Split Format Projector Precision: Standalone F16 GGUF (mmproj) preserving full vision patch fidelity
Quantization Engine: llama.cpp k-quants + custom ComfyUI-GGUF-Loader hybrid tensor packer

Contains both the Language Backbone (blk.*) and Vision Tower (model.visual.*) in a single GGUF file.

File Name	LLM Precision	Vision Precision	File Size	Description
`qwen3vl_8b_q4_k_m_vision_q8_0.gguf`	Q4_K_M	Q8_0 / F32	5.62 GB	All-in-one hybrid GGUF. Best speed/VRAM balance for `ComfyUI-GGUF-Loader` (no external `mmproj` needed).
`qwen3vl_8b_q5_k_m_vision_q8_0.gguf`	Q5_K_M	Q8_0 / F32	~6.39 GB	All-in-one hybrid GGUF. Higher-precision text backbone paired with `Q8_0` vision & DeepStack mergers.

For llama.cpp, llama-server, or split-loader workflows.

File Name	Precision	File Size	Description
`mmproj-qwen3vl-8b-f16.gguf`	F16	1.08 GB	Standalone multimodal vision projector (pair with split 8B files below)
`qwen3vl-8b-Q4_K_M.gguf`	Q4_K_M	4.68 GB	Optimal balance of generation speed, memory, and accuracy
`qwen3vl-8b-Q5_K_M.gguf`	Q5_K_M	5.45 GB	Balanced higher-precision text backbone
`qwen3vl-8b-Q6_K.gguf`	Q6_K	6.26 GB	Near-lossless language reasoning and instruction following

File Name	Precision	File Size	Description
`mmproj-qwen3vl-4b-f16.gguf`	F16	0.78 GB	Standalone 4B Multimodal vision projector
`qwen3vl-4b-Q4_K_M.gguf`	Q4_K_M	2.33 GB	Ultra-lightweight edge quant for low-memory environments
`qwen3vl-4b-Q5_K_M.gguf`	Q5_K_M	2.69 GB	Higher-precision 4B variant

Place your desired language GGUF (e.g., qwen3vl-8b-Q4_K_M.gguf) into:
```
ComfyUI/models/unet/
# or
ComfyUI/models/text_encoders/
```
Place the matching projector (e.g., mmproj-qwen3vl-8b-f16.gguf) into:
```
ComfyUI/models/clip_vision/
```

Load the model using your split GGUF loader or via llama.cpp CLI/Server:

llama-server -m qwen3vl-8b-Q4_K_M.gguf --mmproj mmproj-qwen3vl-8b-f16.gguf

Both qwen3vl_8b_q4_k_m_vision_q8_0.gguf and qwen3vl_8b_q5_k_m_vision_q8_0.gguf have been validated end-to-end:

750 Total Tensors: 399 LLM tensors (Q4_K_M or Q5_K_M) + 351 Vision tensors (Q8_0 weights + F32 norms/biases).
5D Patch Embedding Metadata: Includes comfy.gguf.orig_shape.model.visual.patch_embed.proj.weight = [1152, 3, 2, 16, 16] for 3D temporal-spatial patch projection.
DeepStack Mergers: All 3 DeepStack spatial mergers ([4096, 4608]) and final vision-to-LLM merger verified with zero NaN/Inf values during multimodal forward passes.

Alibaba Cloud Qwen Team for the foundational Qwen3-VL models.
Comfy-Org for packaging the unified BF16 safetensors weights.
Georgi Gerganov and the GGML/llama.cpp contributors for GGUF architecture and quantization tooling.