DeepSeek-V4.1-Flash-1017-NVFP4

This is the first NVFP4 checkpoint of deepseek-ai/DeepSeek-V4.1-Flash-1017, in the same tensor layout and config schema as nvidia/DeepSeek-V4.1-Flash-NVFP4.

• Routed experts: a lossless MXFP4 → NVFP4 transcode of the 1017 weights. The 4-bit codes are untouched; only the block scales are re-expressed. The per-projection activation scales are taken from NVIDIA's calibration of V4.1-Flash.
• Engram tables: stored as MXFP4, which is lossy (cosine 0.9934 vs the FP8 source). This is the same method and format as NVIDIA's.
• Everything else: byte-identical to upstream, including the MTP/DSpark experts.

The size goes from 475.24 GiB to 399.51 GiB.

What's in the checkpoint

Expert tensors. <name>.weight is uint8 [out, in/2]: E2M1 codes, two per byte, with the even column in the low nibble. <name>.weight_scale is E4M3 [out, in/16] and <name>.weight_scale_2 is an FP32 scalar, so that:

w[r, c] = E2M1[code[r, c]] * weight_scale[r, c // 16] * weight_scale_2

weight_scale_2 is 2^(largest source block exponent − 8). It is shared by w1 and w3 of an expert, so the largest block scale lands on E4M3 256. This is the rule NVIDIA's checkpoint uses; we confirmed it on sampled V4.1 experts, where their conversion also reproduces the source exactly. <name>.input_scale is the FP32 static activation scale for W4A4.

Engram tensors. layers.{1,14}.engram.embed.weight is uint8 [rows, 128], packed the same way, and .scale is E8M0 [rows, 8], one scale per 32 values.

Why a transcode, not a re-quantization

DeepSeek ships the routed experts in MXFP4. NVFP4's 16-element blocks are a strict refinement of MXFP4's 32-element blocks, and every E8M0 power-of-two scale is exactly representable as E4M3 × a power-of-two global scale. So the same 4-bit codes map to NVFP4 with zero error. Calibrated NVFP4 recipes choose weight scales to minimise error against a higher-precision reference. Examples are Four Over Six's per-block 4-or-6 scale selection in NVIDIA Model Optimizer, or MSE scale sweeps. Here the reference is the shipped 4-bit weights, which the transcode already reproduces exactly. A search can at best match it; any other choice of codes or scales adds error.

Activation scales

The W4A4 experts need one static input_scale per projection. These 46,080 values come from nvidia/DeepSeek-V4.1-Flash-NVFP4 (revision c48dca01). That is NVIDIA's Model Optimizer calibration of V4.1-Flash, of which 1017 is DeepSeek's update. The weights are 1017's.

Six experts received no calibration tokens in NVIDIA's run. For all three of their projections, NVIDIA's checkpoint carries the model-wide maximum instead (their card: "18 of 46,080 projection entries used fallback scales"). We handle these 18 entries as follows:

• w1 / w3 take the largest calibrated value among the other 383 experts in the same layer and projection. All experts in a layer see the same hidden states, so this is still a conservative estimate.
• w2 keeps NVIDIA's value. Its input is the expert's own SwiGLU output, so the other experts say nothing about it, and an overestimate cannot clip.

All other 46,068 values are NVIDIA's, unchanged. A calibration on 1017 itself will replace all of the lifted scales.

Verification

Every number below was measured on this checkpoint.

How it was made:

• The re-packer works one shard at a time. Shards are read with pread(), not memory-mapped, so the 94 GiB Engram tables are streamed and never held in memory.
• Experts keep their nibbles unchanged. Their E8M0 scales are split into E4M3 block scales and a shared power-of-two FP32 global scale.
• Engram scales are recomputed as the smallest power of two ≥ amax/6 per 32 values, so no value clips. Values are rounded to the nearest E2M1 value, with ties toward zero.
• The E2M1 table and nibble order come from DeepSeek's own inference/convert.py.
• The scripts are included: dsv41_fp4_stream.py (re-packer; this build used --global-rule nvidia --keep-mtp --input-scales), finalize_repo.py (index and config), verify_transcode.py (bit-exactness and Engram cosine), check_repo.py (index, header and byte-identity checks), and test_engram_path.py / test_engram_shard.py (cpu/cuda agreement and container round-trip).

Serving

This repository has the same tensor layout and config schema as nvidia/DeepSeek-V4.1-Flash-NVFP4. We checked that every tensor name, dtype and shape matches, and config.json and hf_quant_config.json are identical. Load it the way NVIDIA documents for theirs. Their card states:

• The MXFP4 Engram tables need a vLLM build that includes vllm-project/vllm#61530. The vllm/vllm-openai:deepseekv41-flash-0909 image predates it.
• The MXFP4 Engram tables are not supported in SGLang yet.
• Their tested hardware is NVIDIA GB300 (Blackwell).

NVIDIA's documented vLLM command, with this repository's ID:

vllm serve LibertAIDAI/DeepSeek-V4.1-Flash-1017-NVFP4 \
    --tensor-parallel-size 4 \
    --data-parallel-size 1 \
    --tokenizer-mode deepseek_v41 \
    --reasoning-parser deepseek_v41 \
    --language-model-only \
    --max-model-len 1048576 \
    --max-num-seqs 32 \
    --max-num-batched-tokens 8192 \
    --enable-chunked-prefill \
    --no-enable-prefix-caching \
    --model-loader-extra-config '{"enable_multithread_load": true, "num_threads": 128}'

Sizes for planning: the whole checkpoint is 399.51 GiB. The two Engram tables are 97.28 GiB of that; NVIDIA's card notes that vLLM keeps them in pinned host memory by default. Everything else is 302.23 GiB.

DeepSeek's bundled inference/ code, included unmodified, does not load this layout. Its convert.py converts only int8 experts paired with <name>.scale, and its ParallelEngramEmbedding allocates the tables as float8_e4m3fn [rows, 256].

Upstream ships no Jinja chat template. DeepSeek's prompt-encoding reference is in encoding/, copied unchanged.

Engram

Engram is a hashed n-gram gather, not a GEMM. With engram_layer_ids [1, 14], engram_max_ngram_size 4 and engram_n_heads 8, each token reads 3 n-gram sizes × 8 heads × 2 layers = 48 rows.

To load a row: unpack 256 E2M1 values from 128 bytes, then multiply each group of 32 by its E8M0 scale (2^(e − 127)). config.json declares the format with the keys vLLM PR #61530 reads: engram_embed_quant: "mxfp4", engram_embed_block_size: 32, engram_embed_scale_dtype: "ue8m0" and engram_embed_layers: [1, 14].

The method is NVIDIA's. On the original V4.1-Flash, our Engram quantizer reproduced the MXFP4 tables in nvidia/DeepSeek-V4.1-Flash-NVFP4 byte for byte: 65,536 of 65,536 sampled rows across both tables, weights and scales.

About the base model

DeepSeek describes V4.1-Flash-1017 as a minor update to DeepSeek-V4.1-Flash, with gains in reasoning, coding, agentic and visual tasks, and shorter outputs. Its benchmark table is on the upstream model card; DeepSeek measured it on the upstream weights.

The architecture is the same as V4.1-Flash: upstream config.json is identical, and all the weights are new. It is a multimodal MoE with 40 layers, 384 routed experts + 1 shared expert, 6 routed experts per token, Engram tables at layers 1 and 14, 3 MTP layers, and a 1M-token context.

Credits

• DeepSeek made the model and released it under the MIT license. Their notice, Copyright (c) 2023 DeepSeek, is kept in LICENSE. All model credit is theirs.
• NVIDIA designed the layout this repository follows. The ModelOpt config schema, config.json and hf_quant_config.json are taken verbatim from nvidia/DeepSeek-V4.1-Flash-NVFP4, as are the activation scales (their V4.1-Flash calibration, with the 12 replacements above) and the MXFP4 Engram format. The weights in this repository come from the 1017 release, not from NVIDIA.

License

MIT, inherited from the base model. See LICENSE.

---

Published by LibertAI. Other DeepSeek-V4.1 checkpoints from LibertAIDAI:

• DeepSeek-V4.1-Flash-NVFP4
• DeepSeek-V4.1-Flash-REAP-272E
• DeepSeek-V4.1-Flash-REAP-256E