DeepSeek V4 Flash 0731, GGUF
DeepSeek V4 Flash 0731 is a large-scale Mixture-of-Experts model optimized for ultra-long-context reasoning, efficient inference, and enhanced agentic capabilities. Its architecture combines CSA and HCA attention with manifold-constrained hyper-connections. It features FP4 expert weights, FP8 quantization, and a speculative decoding module for faster generation.
This repository holds Vultr's GGUF quantizations of it. They are hybrid quantizations: rather than applying one precision to the whole network, each layer's precision is chosen from a measured importance score, so the bits go where they change the output most.
Model Overview
| Attribute | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash-0731 |
| Architecture | deepseek4, Mixture-of-Experts |
| Parameters | 284B total |
| Active parameters | approximately 13B |
| Layers | 43 |
| Hidden size | 4,096 |
| Attention heads | 64 |
| Experts | 256 routed, 6 selected per token |
| Context length | 1M tokens |
| Vocabulary | 129,280 |
| License | MIT |
| Attribute | Value |
|---|---|
| Base model | deepseek-ai/DeepSeek-V4-Flash-0731 |
| Architecture | deepseek4, Mixture-of-Experts |
| Parameters | 284B total |
| Active parameters | approximately 13B |
| Layers | 43 |
| Hidden size | 4,096 |
| Attention heads | 64 |
| Experts | 256 routed, 6 selected per token |
| Context length | 1M tokens |
| Vocabulary | 129,280 |
| License | MIT |
Quantizations
| File | Type | Size | KLD vs original |
|---|---|---|---|
| DeepSeek-V4-Flash-0731-MXFP4-*.gguf (2 parts) | MXFP4 | 154.99 GB | 0.0132 |
| DeepSeek-V4-Flash-0731-IQ3_M.gguf | IQ3_M | 136.11 GB | 0.1423 |
| DeepSeek-V4-Flash-0731-IQ3_S.gguf | IQ3_S | 134.68 GB | 0.1445 |
| File | Type | Size | KLD vs original |
|---|---|---|---|
| DeepSeek-V4-Flash-0731-MXFP4-*.gguf (2 parts) | MXFP4 | 154.99 GB | 0.0132 |
| DeepSeek-V4-Flash-0731-IQ3_M.gguf | IQ3_M | 136.11 GB | 0.1423 |
| DeepSeek-V4-Flash-0731-IQ3_S.gguf | IQ3_S | 134.68 GB | 0.1445 |
KL divergence measures how far a quantized model's next-token probabilities drift from those of the original model as released (its routed experts are already in 4-bit MXFP4), averaged over every token of the evaluation text. The lower the value, the closer the quantized model is to the original; zero would mean identical output.
Evaluation
Every variant in this repository was evaluated against the original model as released (its routed experts are already in 4-bit MXFP4) for perplexity, KL divergence and top-token agreement, under identical conditions. The full results are maintained on the model's page in the Vultr Model Library:
https://docs.vultr.com/models/deepseek-v4/deepseek-v4-flash-0731?tab=quantized
That page is the reference for these files. The summary above is an excerpt from it.
HellaSwag and WinoGrande are not included: llama.cpp cannot yet score multiple-choice tasks on this architecture.
Run
llama-server -hf vultr/DeepSeek-V4-Flash-0731-GGUF:MXFP4
The model's chat template is embedded in every file, so no extra prompt formatting is needed.
The repository also includes dspark-DeepSeek-V4-Flash-0731-Q8_0.gguf, the model's DSpark draft model, for optional speculative decoding with any quant above:
llama-server -hf vultr/DeepSeek-V4-Flash-0731-GGUF:MXFP4 --spec-type draft-dspark
License
These files are derived from deepseek-ai/DeepSeek-V4-Flash-0731 and are distributed under the same license, MIT. The model is the work of DeepSeek; Vultr produced the quantizations.