DeepSeek-V4.1-Flash-1017 · dsx + Unsloth UD-IQ1_M experts

DeepSeek-V4.1-Flash-1017 (552B MoE + 196B Engram) in one 305 GB download for two RTX 3090s and 128 GB of RAM. The routed experts are Unsloth's UD-IQ1_M tensors, unchanged. The dense weights (Q8_0), router and vision tower come from dsx's 2-bit build, and the Engram table is the official FP8 table at Q6_K, inside the GGUF together with the tokenizer. Runs on dsx, a Rust + CUDA engine for this model.

In short: this build takes only the routed experts from Unsloth's UD-IQ1_M, byte for byte, and replaces everything else: the attention, shared-expert and LM-head weights come from dsx at Q8_0 (Unsloth uses Q6_K / Q5_K), and Engram is DeepSeek's FP8 table at Q6_K instead of IQ4_NL, so the same experts score slightly better (KL 0.269 vs 0.275). dsx differs from llama.cpp in how it runs a model this size on a desktop: it keeps the experts each conversation uses most in VRAM and swaps them as it changes, computes the rest on the CPU straight from RAM in parallel with the GPUs, has the CPU and GPUs signal each other through mapped-memory flags instead of host synchronization within a token, and uses decode kernels made for this model's attention and expert shapes (optionally letting the GPU run ahead of the CPU's lowest-ranked experts). Same experts, same machine: 27-29.5 tok/s in dsx, 12.0 in llama.cpp (both after a 3K-token prompt).

Which build

All four run on dsx and were measured on the same eval (Lucebox 1017, 8 x 1,024 tokens against the FP8 model's top-256 log-probs).

At a glance

What differs from Unsloth's GGUF and from dsx 2-bit

The only shared part is the routed experts. Everything else, and the engine, comes from dsx.

Quality

Measured on Lucebox's 1017 eval set: 8 × 1024 tokens against the FP8 model's top-256 log-probs.

The experts are the same bytes in both engines. dsx scores slightly better than llama.cpp because it uses Q8_0 dense weights and the Engram table at Q6_K, where Unsloth's GGUF has lower-bit dense weights and Engram in IQ4_NL.

The Engram table format, everything else the same (this build's experts and dense weights in dsx):

--defer-rank 3 adds the CPU's lower-ranked experts one layer late (after KTransformers), for +14% decode speed.

Speed

Greedy decoding, exact routing, 300 generated tokens, 2x RTX 3090 (230 W) + 128 GB DDR4-3200.

Run

hf download lxe/DeepSeek-V4.1-Flash-1017-dsx-ud --local-dir dsx-ud

git clone https://github.com/lxe/dsx && cd dsx
CUDA_ARCHS=86 cargo build --release -p dsx --features cuda      # 86 = RTX 30xx, 120 = RTX 50xx

./target/release/dsx-server --model ../dsx-ud/DeepSeek-V4.1-Flash-1017-dsx-ud-00001-of-00005.gguf \
  --host 0.0.0.0 --port 8090 --ctx 131072 --arena-gb 110 --rank ../dsx-ud/expert-rank.json --defer-rank 3

The Engram table and the tokenizer are read from the GGUF; no --engram or --tokenizer needed.

Leave out --defer-rank 3 for the best quality. Open http://localhost:8090/ for the web UI, or point any OpenAI-compatible client at http://localhost:8090/v1.

Hardware

VRAM + RAM should hold ~135 GB of experts:

The Engram table stays on disk and is read a few rows per token.

How it was made

quant/hybrid_gguf.py in dsx writes one GGUF with every tensor and key of dsx's v4b build except its routed experts, plus the blk.N.ffn_{gate,up,down}_exps.weight tensors of Unsloth's UD-IQ1_M (revision 60ad3651). quant/engram/engram_gguf.py turns the official FP8 Engram tables into one Q6_K tensor (per_layer_token_embd.weight, llama.cpp's layout), quantized by ggml's own Q6_K code from the bf16-rounded FP8 values. dsx split --shards 5 --add engram-q6k.gguf --tokenizer-json tokenizer.json writes the model in 4 shards, the Engram table as the 5th, and the official tokenizer.json as the dsx.tokenizer_json key. The experts are unrotated, so the build has no dsx.expert_rot key. dsx runs the four IQ formats with its own CPU kernels and GPU tiles.

Files

Credits

Unsloth for the UD-IQ1_M experts (GGUF on Hugging Face, MIT) · DeepSeek for the model (MIT) · llama.cpp / ggml for the IQ formats and codebook tables (MIT) · Xolotl / Strata for the V4.1 GGUF format (MIT) · Lucebox for the eval set · KTransformers (Chen et al., SOSP '25) for expert deferral.