DeepSeek-V4.1-Flash-1017 for Basalt
Website: basalt.jgulfo.com
Model file for Basalt, a local inference engine for NVIDIA Blackwell GPUs (sm_120) with an OpenAI- and Anthropic-compatible API. Basalt runs this model on one Blackwell card, optionally a second card as a peer expert tier, and system RAM for the experts that do not fit in VRAM.
Packs, each one self-contained file holding an Unsloth quantization (Dynamic V3.1) of deepseek-ai/DeepSeek-V4.1-Flash-1017: weights, the Engram table, the vision projector, the tokenizer and the DSpark drafter. They need Basalt 0.3.0 or later.
| file | size | experts |
|---|---|---|
DeepSeek-V4.1-Flash-1017-UD-IQ1_M.basalt |
260 GB | UD-IQ1_M: faster, more of the model fits in RAM |
DeepSeek-V4.1-Flash-1017-UD-IQ2_XXS.basalt |
285 GB | UD-IQ2_XXS: closer to BF16, slower when experts spill to disk |
- Vision: images work with
--vision dynamic, encoded inside the engine from the file's projector. - Context: up to 1,048,576 tokens (
--max-context). - Drafter: DSpark drafts by default in windows of 3, keeping each stage's 32 most routed experts (2.4 GB of VRAM).
Drafted tokens are verified, so the output is the model's own.
--spec 0turns it off. Numbers and when it pays: what it costs.
Speed
Single stream, RTX 5090 at 400 W + RTX 5060 Ti at 180 W as a peer, tokens/s. Numbers move with every release.
| pack | prose | structured | chat | prefill, 64K prompt |
|---|---|---|---|---|
| UD-IQ1_M | 94 | 179 | 86 | 2,782 |
| UD-IQ2_XXS | 45 | 102 | 40 | 1,363 |
DSpark drafting on (the default). Long agentic coding sessions averaged about 85 tok/s.
Quality
Measured with basalt-bench --quality: 100 chunks of wikitext-2 (512 tokens each), scored through the decode path
against a llama.cpp BF16 reference of the original model (reference PPL 2.731 on the scored rows), made by
@giveen. RTX 5090 at 400 W + RTX 5060 Ti at 180 W.
| pack | PPL | KLD mean | same top-1 |
|---|---|---|---|
| UD-IQ1_M | 3.033 | 0.278 | 83.88% |
| UD-IQ2_XXS | 3.107 | 0.241 | 85.15% |
Basalt does not requantize the weights: every tensor is stored as the GGUF publishes it. How each stage rounds is in the precision page.
Run
Build the engine as described in the Basalt README, then:
hf download jesdga/DeepSeek-V4.1-Flash-1017-Basalt DeepSeek-V4.1-Flash-1017-UD-IQ1_M.basalt --local-dir models # or the UD-IQ2_XXS file
bin/basalt serve models/DeepSeek-V4.1-Flash-1017-UD-IQ1_M.basalt --vision dynamic
Add --gpu 0,1 to use a second card and --max-context N for a longer context. The engine reads the model's chat
format, tool-call syntax and thinking switch from the file. More in
DeepSeek-V4.1 on Basalt and
models and the router.
Sources
- Quantization: Unsloth, UD-IQ1_M and UD-IQ2_XXS (Dynamic V3.1); the Engram table, the vision projector (BF16) and the DSpark drafter (Q8_0) come from the same release.
- Model: deepseek-ai/DeepSeek-V4.1-Flash-1017, DeepSeek.
- Basalt is a fork of Strata (MIT).
License
The model is released by DeepSeek under the MIT License, included as LICENSE. Unsloth publishes its quantization under MIT as well. The Basalt engine is MIT.