GLM-4.6V-Flash vision tower (mmproj GGUF)

The vision half of zai-org/GLM-4.6V-Flash, packaged as a llama.cpp multimodal projector. This is the counterpart to sartajbhuvaji/GLM-4.6-Flash-text-GGUF, which is the same model with vision removed.

892,498,432 params, 24-layer ViT + merger, clip.projector_type: glm4v.

Why there is no Q4_K_M

Vision projectors are not quantized the way language models are. llama-quantize and the K-quant ladder (Q4_K_M, Q5_K_M, Q6_K, …) apply to the text GGUF; mmproj files are produced directly by convert_hf_to_gguf.py --mmproj, whose only output types are f32, bf16, f16, and q8_0.

This matches what other publishers ship. unsloth's Gemma 3 and Gemma 4 mmproj files are BF16/F16/F32 only, and ggml-org's Qwen2.5-VL mmproj is Q8_0/F16. Vision encoders are more sensitive to low-bit quantization than decoders and the tooling reflects that. Q8_0 at 1.03 GB is as small as this gets.

Usage

An mmproj file does nothing on its own. It has to be paired with a text GGUF.

llama-mtmd-cli \
  -m GLM-4.6-Flash-text-Q4_K_M.gguf \
  --mmproj mmproj-GLM-4.6V-Flash-F16.gguf \
  --image photo.jpg \
  -p "Describe this image." -n 400 -st

Get the text half from sartajbhuvaji/GLM-4.6-Flash-text-GGUF:

hf download sartajbhuvaji/GLM-4.6-Flash-text-GGUF \
  GLM-4.6-Flash-text-Q4_K_M.gguf --local-dir .
hf download sartajbhuvaji/GLM-4.6V-Flash-mmproj-GGUF \
  mmproj-GLM-4.6V-Flash-F16.gguf --local-dir .

-st (--single-turn) matters for scripted use. Without it llama.cpp drops into interactive mode and waits on stdin.

Pairing

The text repo is the vision-removed model (Glm4ForCausalLM, arch glm4). llama.cpp has no separate GLM4V model architecture, only CHATGLM, GLM4 and GLM4_MOE, with glm4v existing as a projector type. So GLM-4V's text tower and this text-only extraction both convert to arch glm4, and the metadata lines up.

This pairing was verified from GGUF metadata, not by running end-to-end multimodal inference. The build machine had 7 GiB of RAM, and Q4_K_M (6.17 GB) plus a projector does not fit. If you hit a mismatch, pair against a text GGUF converted directly from zai-org/GLM-4.6V-Flash instead, and please open a discussion so this note can be corrected.

Contents

Derived from the 181 tensors under model.visual.* in the source checkpoint, exported as 182 GGUF tensors (v.* for the ViT, mm.* for the merger).

Config: image_size 336, patch_size 14, projection_dim 4096, block_count 24, head_count 12, feed_forward_length 13696, SiLU activation.

Provenance

Exported with convert_hf_to_gguf.py --mmproj from llama.cpp at master. Derived from zai-org/GLM-4.6V-Flash (MIT); this repo is MIT.

Related: GLM-4.6-Flash-text (bf16 text-only) · GLM-4.6-Flash-text-GGUF (text quants).