DeepSeek-V4-Flash-Vision-Exp-VQ-3.2bpw

106.4 GiB text weights. Text and image input. Vision tower +0.87 GiB; DSpark MTP draft head +10.53 GiB; full download 117.8 GiB.

A vector-quantized build of DeepSeek-V4-Flash-Vision-Exp, the vision variant of DeepSeek-V4-Flash. The official release is about 156 GiB on disk. The vision tower ships exact, so image input works as in the release.

It is about 32% smaller than the release and keeps image input.

Runtime note. This architecture (deepseek_v4 with vision) is not in stock mlx-lm, and plain transformers does not register it. Stock mlx-lm and plain transformers cannot load this model. It runs on Knurlogic 0.1.3 or newer, which ships the architecture. See Run it.

Changelog

2026-10-04 — First release

First release, with the vision tower and the DSpark draft head.

Measured results

KL to the exact teacher, cached top-64, 12288 tokens per corpus, on all three house corpora (prose, code, literary), every row scored on the same cache and the same positions. Millinats per token; lower is closer to the teacher.

This build beats the uniform one on all three corpora (paired t -8.6, -3.6, -7.8) for +2.3 GiB. Scored on Knurlogic 0.1.3, which matches DeepSeek's reference numerics.

Task benchmarks

Task benchmarks have not been run on this build.

Hardware

Fits a single 128 GB Apple Silicon Mac without the draft head; with it, run it split across two Macs with Knurlogic.

Speculative decoding (MTP)

Ships DeepSeek's DSpark draft head (mtp-head-dspark-mxfp4.safetensors). Knurlogic finds it beside the weights and drafts with it.

Run it

Requires a Mac with Apple Silicon (macOS, MLX).

Stock mlx-lm and plain transformers cannot load this model (see the runtime note above). Use Knurlogic 0.1.3 or newer, which ships the architecture, works out the settings and checks the model fits before loading it:

pip install knurlogic
hf download TheDrainFlorist/DeepSeek-V4-Flash-Vision-Exp-VQ-3.2bpw --local-dir ~/Knurlogic/Models/DeepSeek-V4-Flash-Vision-Exp-VQ-3.2bpw
knurlogic serve ~/Knurlogic/Models/DeepSeek-V4-Flash-Vision-Exp-VQ-3.2bpw

Chat at http://127.0.0.1:8080/, or point any OpenAI-, Anthropic- or Ollama-compatible client at it (model local). Attach an image in the chat page, or send one through the API. knurlogic ui opens a page with a Launch button for every model on disk instead.

The VQ runtime ships inside this repo as model.py (declared by model_file in config.json), and Knurlogic runs the file the model ships.

Methodology

The routed experts (about 97% of the parameters) are vector-quantized: each small group of weights stores one index into a learned codebook, fit in pure weight space with no calibration data. The middle layers, where this model measured the most damage, get a larger codebook. Attention and the shared expert are 8-bit; the vision tower is exact. Evaluated by KL to the exact teacher on three corpora, with paired comparisons on identical positions.

Verification

Generated through Knurlogic 0.1.3 split across two Macs (M3 Ultra 96 GB and M4 Max 128 GB): a coherent text reply, and given a screenshot, the model read its text correctly.

Limitations

• Needs a runtime that ships the deepseek_v4 vision architecture (Knurlogic 0.1.3 or newer). Stock mlx-lm and plain transformers cannot load it.
• KL was measured on text only; image input was checked by smoke only.
• This is a thinking model: it spends tokens reasoning before answering. Budget max_tokens accordingly.
• No absolute speed figure and no task benchmarks are published for this build.

Paper

The method, the full model ladder, the negative results, and the measurement rules behind every number here: Data-Free Vector Quantization Beats Affine Quantization at Matched Bytes Below 6 Bits (CC BY 4.0) · code: VQLab · web version: Space

Support this work

VQLab and these artifacts are built and released independently — the fits, the measurement harness and the published rungs are one person's compute and time. If they are useful to you:

• Sponsor: github.com/sponsors/noahzelezny
• Contract work: available for quantization and on-device inference work on Apple silicon — custom rungs, per-layer allocation for your model, or getting a checkpoint to run well on a Mac. Contact: hello@thedrainflorist.com

Provenance

Base model: deepseek-ai/DeepSeek-V4-Flash-Vision-Exp — MIT. This is a quantized derivative and inherits that license; using it means accepting the base model's terms.

Build record: vqlab_provenance.json (id e611961ab810); lineage: graft-extras (2026-10-03).

Built with MLX and VQLab.