DeepSeek-V4.1-Flash-1017 RANMA EXL3 GGUF

[!WARNING] Requires ranma.cpp ranma_20261013 or newer (github.com/saturnsky/ranma.cpp). These files use fork-specific tensor types (256 and above). They will not open in upstream llama.cpp, LM Studio, Ollama or other llama.cpp-based tools.

ranma.cpp is a llama.cpp fork developed with RDNA4 as its main target. It runs MoE models on a three-tier expert cache: VRAM, host RAM and SSD. It supports EXL3 quantization to keep memory traffic as low as possible at a given quality.

Source and credit

This repository repackages LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw (revision c178a01) as GGUF. The model is deepseek-ai/DeepSeek-V4.1-Flash-1017; the EXL3 quantization was made by LibertAIDAI, following the tensor layout of Mia-AiLab's DeepSeek-V4.1-Flash EXL3 release. The weights were not re-quantized: the trellis-coded tensors are carried over bit for bit, only wrapped in the GGUF container and split into shards. All credit goes to LibertAIDAI, Mia-AiLab and the ExLlamaV3 project. Thank you.

Each folder has a short README. manifest.json lists the size and SHA-256 of every file. Not included: the vision encoder (no image input) and the checkpoint's MTP layers (speculative decoding uses the DSpark draft model).

Running

1. Download the model folder from this repository and the 1017 engram tables (202.8 GB, needed, not in this repository) from SaturnHeaven/DeepSeek-V4.1-Flash-1017-RANMA-Engram-GGUF. The 1017 checkpoint has new tables: do not use the tables of the earlier DeepSeek-V4.1-Flash.
2. Give -m only the first shard, DeepSeek-V4.1-Flash-1017-EXL3-2.9bpw-h6-00001-of-00006.gguf; the other shards are found by name.
3. Give --engram-tables the first file of the table set, DeepSeek-V4.1-Flash-1017-engram-MXFP8-00001-of-00007.gguf.

Context, L1/L2 sizes and the draft model for each hardware configuration are in the ranma.cpp recipes. They are examples: choose for your own use and system.

Benchmarks

These rows were measured with the 3.0 bpw conversion of the earlier DeepSeek-V4.1-Flash (DeepSeek-V4.1-Flash-EXL3-3.0bpw-RANMA-EXL3-GGUF), not with this one. This 2.9 bpw conversion has the same architecture and moves fewer bytes per token, so it runs somewhat faster than these numbers.

ranma_20261018 on a Radeon AI PRO R9700 (32 GB) with layer-major prefill (--layer-mode on) and n-gram drafting (--spec-type ngram-varlen): server decode, warm, one slot, the model's recommended sampling, 12 coding and 17 Korean roleplay requests, and prompt processing of 32768 tokens. The host RAM sizes were reproduced on a 224 GB machine by occupying memory. Settings and the other statistics: benchmark record.

License

Same as the original model: MIT, see the original repository.