DeepSeek V4 Flash Vision Exp, GGUF
DeepSeek V4 Flash Vision Exp is a multimodal Mixture-of-Experts model designed for visual understanding and multimodal agentic workflows while maintaining strong text-only agent performance. The model employs DeepSeek sparse attention. It integrates a vision encoder for enhanced multimodal reasoning and visual processing.
This repository holds Vultr's GGUF quantizations of it. They are hybrid quantizations: rather than applying one precision to the whole network, each layer's precision is chosen from a measured importance score, so the bits go where they change the output most.
Model Overview
| Attribute | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash-Vision-Exp |
| Architecture | deepseek4, Mixture-of-Experts |
| Parameters | 284B total |
| Active parameters | 13B |
| Layers | 43 |
| MTP layers | 3 |
| Vision encoder layers | 32 |
| Hidden size | 4,096 |
| Attention heads | 64 |
| Key-value heads | 1 |
| Experts | 256 routed, 6 activated per token, 1 shared |
| Context length | up to 1M tokens |
| Vocabulary | 129,280 |
| License | MIT |
| Attribute | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash-Vision-Exp |
| Architecture | deepseek4, Mixture-of-Experts |
| Parameters | 284B total |
| Active parameters | 13B |
| Layers | 43 |
| MTP layers | 3 |
| Vision encoder layers | 32 |
| Hidden size | 4,096 |
| Attention heads | 64 |
| Key-value heads | 1 |
| Experts | 256 routed, 6 activated per token, 1 shared |
| Context length | up to 1M tokens |
| Vocabulary | 129,280 |
| License | MIT |
Quantizations
| File | Type | Size | KLD vs original |
|---|---|---|---|
| DeepSeek-V4-Flash-Vision-Exp-MXFP4-*.gguf (2 parts) | MXFP4 | 154.99 GB | 0.0107 |
| DeepSeek-V4-Flash-Vision-Exp-IQ3_M.gguf | IQ3_M | 136.11 GB | 0.1095 |
| DeepSeek-V4-Flash-Vision-Exp-IQ3_S.gguf | IQ3_S | 134.68 GB | 0.1111 |
| File | Type | Size | KLD vs original |
|---|---|---|---|
| DeepSeek-V4-Flash-Vision-Exp-MXFP4-*.gguf (2 parts) | MXFP4 | 154.99 GB | 0.0107 |
| DeepSeek-V4-Flash-Vision-Exp-IQ3_M.gguf | IQ3_M | 136.11 GB | 0.1095 |
| DeepSeek-V4-Flash-Vision-Exp-IQ3_S.gguf | IQ3_S | 134.68 GB | 0.1111 |
The vision encoder is in DeepSeek-V4-Flash-Vision-Exp-mmproj-f16.gguf (933 MB). llama.cpp downloads it automatically with the command under Run, so image input works without extra steps.
KL divergence measures how far a quantized model's next-token probabilities drift from those of the original model as released (its routed experts are already in 4-bit MXFP4), averaged over every token of the evaluation text. The lower the value, the closer the quantized model is to the original; zero would mean identical output.
Evaluation
Every variant in this repository was evaluated against the original model as released (its routed experts are already in 4-bit MXFP4) for perplexity, KL divergence and top-token agreement, under identical conditions. The full results are maintained on the model's page in the Vultr Model Library:
https://docs.vultr.com/models/deepseek-v4/deepseek-v4-flash-vision-exp?tab=quantized
That page is the reference for these files. The summary above is an excerpt from it.
HellaSwag and WinoGrande are not included: llama.cpp cannot yet score multiple-choice tasks on this architecture.
Run
llama-server -hf vultr/DeepSeek-V4-Flash-Vision-Exp-GGUF:MXFP4
The model's chat template is embedded in every file, so no extra prompt formatting is needed.
The repository also includes dspark-DeepSeek-V4-Flash-Vision-Exp-Q8_0.gguf, the model's DSpark draft model, for optional speculative decoding with any quant above:
llama-server -hf vultr/DeepSeek-V4-Flash-Vision-Exp-GGUF:MXFP4 --spec-type draft-dspark
License
These files are derived from deepseek-ai/DeepSeek-V4-Flash-Vision-Exp and are distributed under the same license, MIT. The model is the work of DeepSeek; Vultr produced the quantizations.