DeepSeek-V4.1-Flash-1017 (GGUF)

The file names did not change between versions. The 1017 files carry general.version = 1017 in the GGUF header, the previous files have no general.version.

These files are built for mx-llama.cpp, a fork of llama.cpp focused on multi-GPU inference: layer pipeline parallelism, tensor parallelism (-sm tensor) with a custom AllReduce, and DSpark / MTP speculative decoding. Prebuilt AMD / ROCm and NVIDIA / CUDA images are on Docker Hub at mxxm/mx-llama.cpp.

deepseek-ai/DeepSeek-V4.1-Flash-1017 converted to GGUF, plus its DSpark draft stages as a separate sidecar file. These files hold the text model only: the vision tower and its projector are not included, and there is no mmproj. DeepSeek's original model card follows below.

Files

Shards 00009 and 00011 each hold one Engram table (about 97 GiB together). They are read at random during prompt processing and stay on disk rather than in VRAM, so they belong on the fastest storage you have.

The sidecar holds the three mtp.* draft stages, 128 routed experts each with 3 active, plus the DSpark heads. It taps the attention input of target layers 37, 38 and 39. It is a draft model for speculative decoding against the main file and cannot generate on its own.

Running it

The weights occupy roughly 302 GiB once loaded. An example command:

GGML_ENABLE_CUSTOM_AR=1 HSA_FORCE_FINE_GRAIN_PCIE=1 GPU_MAX_HW_QUEUES=8 \
llama-server -m DeepSeek-V4.1-Flash-MXFP4-00001-of-00012.gguf \
  -sm layer -ngl 99 -fit off \
  -c 32768 -b 8192 -ub 256 -fa 1 -lm dio

• Add -md DeepSeek-V4.1-Flash-DSpark-MXFP4.gguf --spec-type draft-dspark --spec-draft-n-max 3 for DSpark speculative decoding.
• -sm tensor -tps N also works, for N of 1, 2, 4 or 8. On ten MI50 over PCIe -sm layer is faster for both prompt processing and generation.
• Chat completions place the reply in reasoning_content.

Quantization

Routed experts are repacked from the checkpoint's own FP4 to MXFP4, a format change rather than a requantization. FP8 block-scaled tensors are dequantized and stored Q8_0. Norms, hyper-connection coefficients and the router bias stay F32. The router, markov and confidence weights stay BF16. No weights were retrained, fine-tuned or otherwise altered.

Checked against the source checkpoint: every declared tensor is present, FP8 tensors dequantize to the source values, the sidecar accounts for all 69 mtp.* tensors, and the Engram hash tables reproduce the reference engram.py derivation.

License

MIT, inherited from the base model. A copy is included in this repository as LICENSE, as its terms require. Copyright (c) 2023 DeepSeek.

---

Original model card

Reproduced from deepseek-ai/DeepSeek-V4.1-Flash-1017, unchanged except that its relative links now point to that repository.

DeepSeek-V4.1-Flash-1017

---

Technical Report 👁️

Introduction

DeepSeek-V4.1-Flash-1017 is a minor update to DeepSeek-V4.1-Flash, delivering substantial improvements across reasoning, coding, agentic tasks, and visual benchmarks. Alongside these gains, the model reduces task execution time and output length, enabling users to complete tasks faster and at lower cost and advancing the performance–cost Pareto frontier across multiple agent harnesses.

† denotes the text-only subset of HLE. We use the Claude Code harness for Terminal-Bench 4.0 and the Mini-SWE-V2 harness for DeepSWE v1.1.

Prompt Encoding

This release does not include a Jinja-format chat template. The encoding folder contains a self-contained Python reference implementation (encoding.py) with test cases for multi-turn conversations, tool calling, thinking mode, numeric reasoning effort, mid-conversation system messages, and interleaved image content.

For production use, we additionally release deepseek-recipe, a set of Rust libraries with Python bindings that provides the same prompt format as a maintained, protocol-aware toolkit. It converts Messages, Chat Completions, and Responses API requests into the Conversation format, encodes them into DeepSeek V4 and V4.1 prompts or token IDs, and parses model output back into complete or streamed responses — covering thinking, tool calls, images, and generation settings. Model inference, tool execution, and HTTP transport are left to the caller.

Minimal Inference

Please refer to the inference folder for instructions on weight conversion and running inference locally.

Recommended sampling parameters:

License

This repository and the model weights are licensed under the MIT License.

Citation

@misc{deepseekai2026deepseekv41flash,
      title={DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression},
      author={DeepSeek-AI},
      year={2026},
}

Contact

If you have any questions, please raise an issue or contact us at service@deepseek.com.