DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder IQ2_XXS, NInfer v3 artifact

A NInfer v3 artifact of ISTA-DASLab's DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder for NInfer-all, the master branch of iamwavecut/ninfer-all. The Coder is an expert-pruned build of DeepSeek-V4.1-Flash-1017: ISTA-DASLab removed 192 of the 384 routed experts of each of the 40 blocks, chosen with RCO to keep code, agentic tool use, long context and vision, and kept the per-tensor quantization of its IQ2_XXS build in the other 192. Six experts are active per token, as in the release. The GGUF assigns a ggml quantization type to every tensor; this artifact keeps each tensor's ggml blocks byte for byte, and the engine multiplies the blocks in place.

Removing half of the experts costs capability outside the domains the expert search was calibrated on; ISTA-DASLab recommends the unpruned releases for general use, and WaveCut/DeepSeek-V4.1-Flash-1017-GSQ-RCO-IQ1_M-NInfer-v3 is NInfer's conversion of one of them.

Stock NInfer builds refuse this file: the model family and the gguf_* formats exist only in NInfer-all. NInfer serves it on one 24 GB GPU with its routed experts in pinned host memory (61.5 GiB) or streamed from this file, with what the IQ1_M artifact offers: up to eight concurrent requests, prompt-prefix reuse, reasoning, DSML tool calls returned as OpenAI tool calls, structured output, images (--vision), DSpark drafts (--spec dspark) and context-lookup drafting. With host experts, CPU threads compute a share of each decode call's missing experts while the GPU copies the rest.

The Engram table is a separate download, the one the IQ1_M artifact reads: WaveCut/DeepSeek-V4.1-Flash-Engram-IQ4_NL-NInfer-v3.

What is inside

component	representation
routed experts, 40 blocks × 192	the GGUF's blocks, one type per block and projection: IQ1_M (44 of the 80 banks), IQ2_XXS (25), IQ2_XS (11); each expert's gate rows are followed by its up rows, so one expert is one contiguous range of bytes
shared experts	Q5_K, as in the GGUF
latent attention	query and latent down-projections Q8_0; query up-projection and the grouped output projections Q5_K
router, compressors, indexers, hyper-connection matrices	FP32, widened from the GGUF's BF16 and F16, which FP32 holds exactly
norms, attention sinks	FP32, as stored
Engram projections	Q5_K; each block's gate vector is the GGUF's stored FP32 gate
token table, output head	Q5_K
DSpark drafter (`dspark` component)	the release's drafter of three blocks (`mtp.0..2`) from beamster's Sushi STQ pack, the same objects as in the IQ1_M artifact: STQ trellis words (`uint16`) and FP16 Hadamard sign vectors, FP32 vectors and the BF16 Markov tables. It drafts for the unpruned model; verification keeps plain decoding's tokens
vision tower (`vision` component)	the release's tower and aligner from the BF16 `mmproj-DeepSeek-V4.1-Flash-1017-BF16.gguf` of ISTA-DASLab's repository: BF16 matrices; the patch embedding, norms, biases and the learned start, newline and end rows of an image's span FP32 (the same BF16 values); the routing bias of image positions (FP32), the Coder GGUF's `blk.N.exp_probs_b_vl.bias` (192 values a block)
Engram table	not in this file: the model's `engram` description names its table by the SHA-256 of its rows (`36ee7d37…6b41a5`), the same table as the IQ1_M artifact's, published once in the table repository
tokenizer, configuration	DeepSeek's files
chat template	NInfer's transcription of the release's `encoding/encoding.py`; the release ships no Jinja template

One file, DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-ninfer-v3.ninfer, of 77,148,393,216 bytes (71.85 GiB), next to its conversion report, SHA256SUMS, NOTICE and LICENSE. The routed experts are 61.50 GiB of it; the GPU keeps the other 5.85 GiB, 3.58 GiB more for the drafter when it runs and 0.90 GiB for the vision tower with --vision. Conversion command (master from commit ab84dcfa0 on, with the gguf Python package installed):

hf download deepseek-ai/DeepSeek-V4.1-Flash-1017 config.json tokenizer.json tokenizer_config.json \
  --revision 1bae0e1088993b9b82de08f59de5be5a34e42170 --local-dir models/deepseek-v4.1/hf
hf download ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-GGUF \
  --revision 54229d38d588a66d1c6cff08af7b281b9326c656 \
  --include 'IQ2_XXS/*.gguf' --include 'DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf' \
  --include 'mmproj-DeepSeek-V4.1-Flash-1017-BF16.gguf' --local-dir models/deepseek-v4.1-coder
hf download beamster/DeepSeek-V4.1-Flash-1017-Sushi-STQ-1.75bpw \
  --revision 07fec972538639726b868f81e89d9d4f5005fe10 --include config.json \
  --include model.safetensors.index.json --include 'model-mtp-*.safetensors' \
  --local-dir models/deepseek-v4.1/stq
python -m tools.convert --model models/deepseek-v4.1/hf --recipe deepseek_v41_gguf \
  --components text,dspark,vision \
  --source gguf=models/deepseek-v4.1-coder/IQ2_XXS/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-00001-of-00006.gguf \
  --source engram=models/deepseek-v4.1-coder/DeepSeek-V4.1-Flash-Engram-IQ4_NL.gguf \
  --source dspark=models/deepseek-v4.1/stq \
  --source vision=models/deepseek-v4.1-coder/mmproj-DeepSeek-V4.1-Flash-1017-BF16.gguf \
  --name DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS \
  --out models/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-ninfer-v3.ninfer --device cpu

This file was converted with that command on November 4 at master 379c0d999, in 3 min 30 s on the 128 threads of a 64-core EPYC 7B13, hashing the Engram GGUF's 110.6 GB included. The converter takes the expert count from the GGUF and reads the Engram GGUF only for the table's digest; the Coder's Engram GGUF and mmproj are the files of ISTA-DASLab's unpruned repository. --components text,dspark leaves the vision tower out and --components text the drafter too.

Running

Download the model and the Engram table, then serve them from a build of master (its README):

hf download WaveCut/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-NInfer-v3 \
  DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-ninfer-v3.ninfer --local-dir models
hf download WaveCut/DeepSeek-V4.1-Flash-Engram-IQ4_NL-NInfer-v3 \
  DeepSeek-V4.1-Flash-Engram-IQ4_NL-ninfer-v3.ninfer --local-dir models
M=models/DeepSeek-V4.1-Flash-1017-GSQ-RCO-Coder-IQ2_XXS-ninfer-v3.ninfer
T=models/DeepSeek-V4.1-Flash-Engram-IQ4_NL-ninfer-v3.ninfer

# One 24 GB GPU and RAM for the experts: 61.5 GiB pinned, the most used experts cached on the GPU.
ninfer-serve $M --ngram-table $T --model-id deepseek-v4.1-flash-coder \
  --expert-residency host --max-context 32768

# One 24 GB GPU and less RAM: the experts stay in the file and stream into a GPU cache.
ninfer-serve $M --ngram-table $T --model-id deepseek-v4.1-flash-coder \
  --expert-residency disk --max-context 32768

# The same, reading images (OpenAI image_url parts, Anthropic image blocks).
ninfer-serve $M --ngram-table $T --model-id deepseek-v4.1-flash-coder \
  --expert-residency host --max-context 32768 --vision

# DSpark drafts, up to five a round.
ninfer-serve $M --ngram-table $T --model-id deepseek-v4.1-flash-coder \
  --expert-residency host --max-context 32768 --spec dspark

With host experts the GPU holds the 5.85 GiB of dense weights, two whole-layer expert banks (3.7 GiB), the request state and an expert cache that takes what is free after startup (11.5 GiB, 1,429 of the 7,680 experts, on the RTX 4090 below with --max-context 16384; --expert-cache-mib sets it). NInfer builds from 44d6dd4ca on start the cache from this model's own routing profile, recorded on 48 coding, agent and reasoning requests; older builds hold only the unpruned model's profile of 384 experts a block, so they fill this model's cache from its first requests. The startup reads the experts with direct I/O and pins them (the server was ready in 22 s on the host below). CPU threads, one per physical core and at most half the usable CPUs (30 on that host, whose container had 61), compute a share of each decode call's missing experts from the pinned copy while the GPU copies the rest, bit for bit the GPU's products. They have no IQ2_XS product, so in the 10 of the 40 blocks with an IQ2_XS bank the GPU copies every miss. --expert-dma-share 1 copies every miss, and disk experts copy every miss by default. Prompts run block by block in spans of up to 16,384 tokens, so each block's missing experts cross the bus once per span; a span borrows up to 1.3 GiB of the expert cache while it runs. With disk experts the host keeps no copy of the banks: each layer's missing experts are read from the file through the OS page cache into bounded transfer buffers, never through a memory map. The Engram rows are read from the table's file the same way (48 rows a token) unless --ngram-residency ram loads the 103 GiB table into RAM. A model started without its table is refused; --no-ngram-table runs it without the Engram blocks, which degrades the output (see the table's card). The caches are BF16. DeepSeek-V4.1-Flash describes the execution.

Thinking is on by default; reasoning_effort low, high (the default) or max writes the release's effort line, and enable_thinking: false or reasoning_effort: "none" turns it off. --spec dspark drafts with the release's DSpark drafter while one request decodes: up to five tokens a round (--draft-tokens 1..5), kept while its confidence is at least --draft-confidence (0.5), verified in one pass of decode rows; the tokens and log probabilities are plain decoding's. The drafter is the release's, trained for the unpruned model; its 3.58 GiB of device memory come out of the expert cache. Without --spec, --lookup-ngram 4 drafts from earlier occurrences in the context and keeps the plain decode's output.

With --vision the server reads images as the release's image_processor.py prepares them: an image takes a span of up to 1,024 prompt positions (a 31 × 31 grid of aligner cells for a square picture; detail: "low" a quarter of that, --vision-max-merged fewer), and the router adds the routing bias of image positions there, the Coder GGUF's values for its 192 experts. The tower's 0.90 GiB of BF16 matrices stay on the GPU, or with --vision-residency overlay its 32 blocks stay in pinned host memory and pass through two device slots (210 MiB of device memory); either comes out of the expert cache. Video is refused.

Quality

ISTA-DASLab's card reports, at reasoning effort 50, SWE-bench Verified 77.00 and LiveCodeBench v6 88.57 for this build, against 81.20 and 89.71 for the release. Those benchmarks were not rerun in NInfer, and no FP64 reference comparison was run for this build. NInfer's matrix kernels take BF16 activations rounded to ggml's q8_1 blocks, as llama.cpp's do, and keep the caches in BF16. On the host below, host experts with every miss copied, with half of them or the automatic share on CPU threads, disk experts and DSpark drafts produced the same 256 greedy tokens for a code request, and the servers of the speed table gave the same answers. With --vision the model read the title, the red circles and the shapes' order in the repository's chart example and described its scene example correctly, with the same text from the resident and the streamed tower.

Speed

November 4, 2026, one rented host: an RTX 4090 (450 W, PCIe 4.0 x16), a 64-core AMD EPYC 7B13 in a container of 61 CPUs and 503 GiB of RAM, the sm_89 build of master at 379c0d999. ninfer-serve --max-context 16384 --max-concurrency 1, one sequence, greedy decoding, thinking off: a 25-token code request answered with 256 tokens and a summary of a 7,775-token C++ file in 128 tokens. The disk row ran after the file was read into the page cache:

experts	experts cached on the card	decode, code answer	decode after 7,775 tokens	prefill of 7,775 tokens	ready to serve
pinned RAM	1,429 of 7,680	37.2 tok/s	39.2 tok/s	1,040 tok/s	22 s
pinned RAM, `--spec dspark`	979	47.9 tok/s	44.3 tok/s	1,020 tok/s	22 s
disk, page cache warm	1,444	15.7 tok/s	17.8 tok/s	774 tok/s	9 s

The drafter accepted 192 of 250 and 75 of 101 drafts. With the CLI and its default context the code request decoded at 31.7 tok/s with every miss copied (--expert-dma-share 1), 37.5 tok/s with half of them on the CPU threads and 37.3 tok/s with the automatic share, which computed 11% of the routed experts on the CPU; the expert cache served 70 to 72% of them. The CLI peaked at 62.2 GiB of resident host memory with host experts and 0.8 GiB with disk experts. One run per row: no spread is measured. These runs share no host, build or prompt with the IQ1_M artifact's measurements and do not compare with them.

Credits and license

Quantized and pruned weights: ISTA-DASLab, DeepSeek-V4.1-Flash-1017 GSQ-RCO Coder, produced with GSQ and RCO from deepseek-ai/DeepSeek-V4.1-Flash-1017. DSpark drafter: beamster's STQ encoding of the release's drafter, mtp.0..2 of beamster/DeepSeek-V4.1-Flash-1017-Sushi-STQ-1.75bpw. All under the MIT License, reproduced in LICENSE. Attribution notices are collected in NOTICE. This repository only re-packs those weights into NInfer's container.