DeepSeek-V4-Flash-0731 NVFP4 GGUF

GGUF conversion of axiomofmind/DeepSeek-V4-Flash-0731-NVFP4, independently calibrated from the official deepseek-ai/DeepSeek-V4-Flash-0731 weights.

Files

• DeepSeek-V4-Flash-0731-NVFP4-Q8attn-00001-of-00004.gguf through 00004: main model, 165.04 GB total
• DeepSeek-V4-Flash-0731-DSpark-MXFP4-Q8attn.gguf: optional DSpark drafter, 10.90 GB

The 129 merged routed-expert tensors use NVFP4. Source FP8 attention/shared-expert linear tensors are Q8_0; embeddings and output head are BF16. The separate DSpark drafter uses nine MXFP4 expert tensors.

For the main model and fully GPU-resident DSpark drafter, 208 GiB or more of aggregate VRAM is recommended. The drafter also requires approximately 10.2 GiB of contiguous free VRAM on one GPU. Actual memory use depends on context length, batch size, and runtime settings.

Runtime requirement

This conversion needs the included llama.cpp-deepseek-v4-nvfp4-runtime.patch. Without it, DeepSeek V4 does not apply the NVFP4 per-expert scales and produces corrupted output. The patch applies to llama.cpp commit 5ecbe1ac17ec0484c5b44af0bd580cdc9c428ed4.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 5ecbe1ac17ec0484c5b44af0bd580cdc9c428ed4
git apply ../llama.cpp-deepseek-v4-nvfp4-runtime.patch

Main-model example:

llama-server -m DeepSeek-V4-Flash-0731-NVFP4-Q8attn-00001-of-00004.gguf \
  -ngl all -c 32768 -fa on --jinja

Main model with the optional GPU-resident DSpark drafter:

llama-server -m DeepSeek-V4-Flash-0731-NVFP4-Q8attn-00001-of-00004.gguf \
  -md DeepSeek-V4-Flash-0731-DSpark-MXFP4-Q8attn.gguf \
  --spec-type draft-dspark --spec-draft-n-max 5 \
  -ngl all -ngld all \
  -c 32768 -fa on --jinja

The main model and DSpark speculative decoding have both been runtime-validated. Performance and draft acceptance depend on hardware, prompt, context, and generation settings.

The GGUF metadata advertises a 1,048,576-token context. Longer contexts require additional KV-cache and compute memory.