DeepSeek-V4.1-Flash-UNCENSORED RANMA EXL3 GGUF

[!WARNING] Requires ranma.cpp ranma_20261013 or newer (github.com/saturnsky/ranma.cpp). These files use fork-specific tensor types (256 and above). They will not open in upstream llama.cpp, LM Studio, Ollama or other llama.cpp-based tools.

ranma.cpp is a llama.cpp fork developed with RDNA4 as its main target. It runs MoE models on a three-tier expert cache: VRAM, host RAM and SSD. It supports EXL3 quantization to keep memory traffic as low as possible at a given quality.

Source and credit

This repository repackages dealignai/DeepSeek-V4.1-Flash-UNCENSORED-EXL3-2.9bpw (revision 8a27b35) as GGUF. The modification of deepseek-ai/DeepSeek-V4.1-Flash (abliteration) and the EXL3 quantization were both made by dealignai. The weights were not re-quantized: the trellis-coded tensors are carried over bit for bit, only wrapped in the GGUF container and split into shards. All credit goes to dealignai and the ExLlamaV3 project. Thank you.

Each folder has a short README. manifest.json lists the size and SHA-256 of every file. Not included: the vision encoder (no image input) and the checkpoint's MTP layers (speculative decoding uses the DSpark draft model).

Running

1. Download the model folder from this repository and the engram tables (202.8 GB, needed, not in this repository) from SaturnHeaven/DeepSeek-V4.1-Flash-RANMA-Engram-GGUF. They are the tables of the base model; the source repository also directs to the base model's tables.
2. Give -m only the first shard, DeepSeek-V4.1-Flash-UNCENSORED-EXL3-2.9bpw-h6-00001-of-00006.gguf; the other shards are found by name.
3. Give --engram-tables the first file of the table set, DeepSeek-V4.1-Flash-engram-MXFP8-00001-of-00007.gguf.

Context, L1/L2 sizes and the draft model for each hardware configuration are in the ranma.cpp recipes. They are examples: choose for your own use and system.

[!NOTE] No benchmark numbers are published for this model.

Benchmarks

These rows were measured with the 3.0 bpw conversion of the original model (DeepSeek-V4.1-Flash-EXL3-3.0bpw-RANMA-EXL3-GGUF), not with this one. This 2.9 bpw conversion has the same architecture and moves fewer bytes per token, so it runs somewhat faster than these numbers.

ranma_20261018 on a Radeon AI PRO R9700 (32 GB) with layer-major prefill (--layer-mode on) and n-gram drafting (--spec-type ngram-varlen): server decode, warm, one slot, the model's recommended sampling, 12 coding and 17 Korean roleplay requests, and prompt processing of 32768 tokens. The host RAM sizes were reproduced on a 224 GB machine by occupying memory. Settings and the other statistics: benchmark record.

License

Same as the original model: MIT, see the original repository.