DeepSeek V4.1 Flash is a 763B-parameter mixture-of-experts model, 510 GB as released. KLX-IQ1_S is 233 GB and runs on four H100 80 GB GPUs with eight 80K-token conversations at once. Standard GGUF, stock llama.cpp, no fork.
| Folder | Size | Mean KL vs. full model ↓ | Pick it for |
|---|---|---|---|
| KLX-IQ1_S | 232.6 GB | 0.082 | running DeepSeek V4.1 Flash on 4× H100 |
You need more memory than the file size, in GPU memory plus system RAM. Scores, the head-to-head and how every number was measured: BENCHMARKS.md.
hf download KarLexAI/DeepSeek-V4.1-Flash-1017-GGUF --include "KLX-IQ1_S/*" --local-dir .
llama-server -m KLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00001-of-00006.gguf --fit on -c 131072 --jinja --reasoning-format deepseek --temp 1.0 --top-p 1.0
Then open http://localhost:8080 and chat. You need a llama.cpp build with DeepSeek V4.1 support (we tested commit
bcb0152). With less GPU memory, --fit on keeps part of the experts in system RAM.
DeepSeek V4.1 Flash also reads images. Download the vision projector (about 1 GB) and add --mmproj:
hf download KarLexAI/DeepSeek-V4.1-Flash-1017-GGUF mmproj-F16.gguf --local-dir .
llama-server -m KLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00001-of-00006.gguf --mmproj mmproj-F16.gguf --fit on -c 131072 --jinja --reasoning-format deepseek --temp 1.0 --top-p 1.0
mmproj-BF16.gguf works the same way (we tested llama.cpp commit 351025ec).
chat_template.jinja.Made by KarLex AI. If this file saves you GPUs, a ❤️ on this repo helps others find it.
Not affiliated with DeepSeek or Unsloth.
KLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00002-of-00006.gguf 65 GBKLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00003-of-00006.gguf 46 GBKLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00005-of-00006.gguf 46 GBKLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00004-of-00006.gguf 45 GBKLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00006-of-00006.gguf 31 GBKLX-IQ1_S/DeepSeek-V4.1-Flash-1017-KLX-IQ1_S-00001-of-00006.gguf 549 MB