DeepSeek V4.1 Flash 1017 GGUF for DwarfStar

Calibrated Q2 weights and the matching vision encoder for DwarfStar, converted from deepseek-ai/DeepSeek-V4.1-Flash-1017 with the V4.1 recipe in gguf-tools.

The language GGUF includes 188.83 GiB of native FP8 Engram tables. DwarfStar reads the needed rows directly from disk, so keep the GGUF on a fast local SSD.

Download and Run

The download targets and the 1017 vision encoder need audreyt/ds4: it pairs a vision encoder with the language GGUF by source revision, while upstream accepts only the original V4.1 encoder.

./download_model.sh ds41f-1017-q2
./download_model.sh ds41f-1017-vision
./ds4 -m gguf/DeepSeek-V4.1-Flash-1017-Q2.gguf \
  --vision gguf/DeepSeek-V4.1-Flash-1017-Vision.gguf --ssd-streaming --ctx 32768

This encoder does not load with the original V4.1 Flash GGUF, and the original encoder does not load with this one.

On a 128 GB M5 Max with --ssd-streaming, ds4-bench measured 202-229 t/s prefill and about 16 t/s generation from 4K to 16K context.

Quantization

IQ2_XXS routed gate/up experts and Q2_K down experts; Q8 attention projections, shared experts and output; F16/F32 tensors elsewhere; native Engram data unchanged. Routed experts use a 32,768-token activation imatrix collected with the DwarfStar Metal collector on a weight-energy bootstrap GGUF; it activated 93.84% of the 15,360 layer-experts, and the rest keep the weight-energy fallback. The GGUF was quantized from the original safetensors, not requantized.

deepseek41_validate_gguf.py --payload --imatrix passed: complete layout, every non-expert tensor, sampled routed experts and Engram rows match the source. The encoder is copied byte for byte from the same checkpoint.

Source: deepseek-ai/DeepSeek-V4.1-Flash-1017 at 1bae0e1088993b9b82de08f59de5be5a34e42170. See the included upstream MIT license.