DeepSeek-V4-Flash Uncensored, text only, KT quant for 128 GB machines
A text-only GGUF of orcarouter/DeepSeek-V4-Flash-Vision-Uncensored, designed for 128 GB of RAM or VRAM. It has no vision encoder, so it reads text only. It's one 98.8 GiB file, and 128 GB holds it with room for up to 1M context.
The source model has its safety alignment removed, and quantization does not restore it. Read the source card's disclaimer before use.
It needs ik_llama.cpp, because mainline llama.cpp doesn't have the IQ2_KT, IQ3_KT, and IQ6_K types. Tested at 0f78b03.
Hugging Face's download widget does not recognize _KT GGUFs. The download link is in the table below.
| File | Bytes | sha256 |
|---|---|---|
DeepSeek-V4-Flash-Uncensored-TextOnly-IQ3_KT.gguf |
106,039,719,328 | 10a47413834de33192b051ba7873c407eaea87a008604c4e9a6bcd5b2c410b75 |
Quantization
Quantized from orcarouter's GGUF (MXFP4 routed experts, Q8_0 attention), with an importance matrix computed on that GGUF.
| Tensors | Type |
|---|---|
| Routed experts | IQ3_KT (88 tensors), IQ2_KT (39), MXFP4 as in the source (2) |
| Attention, shared experts, indexer query projection | IQ6_K |
| Output | Q8_0 |
| Token embeddings, router, compressor, and indexer projections | BF16 |
The 39 IQ2_KT tensors were picked by a per-tensor KLD measurement on English and code.
The importance matrix used 700 chunks of 2048 tokens. The calibration text mixes agentic coding transcripts, English prose (technical and general), Chinese, and code excerpts from nearly every source file in ik_llama.cpp and llama.cpp.
Quality
Against the source GGUF's logits on held-out text, 2048-token context (English 9 chunks, code 10):
| Mean KLD | Same top token | PPL (source) | |
|---|---|---|---|
| English | 0.140 | 87.9% | 4.32 (3.95) |
| Code | 0.088 | 93.2% | 1.95 (1.80) |
Running
llama-server -m DeepSeek-V4-Flash-Uncensored-TextOnly-IQ3_KT.gguf \
-c 262144 -fa on -ngl 999 -np 1 -b 2048 -ub 2048 \
--swa-compress --ctx-checkpoints 8 -ctk q8_0 -ictk q8_0 -fidx --jinja
For 1M context, use -c 1048576 --ctx-checkpoints 2 -ub 1024.
The GGUF embeds ik_llama.cpp's DeepSeek V4 chat template, which matched DeepSeek's reference encoder on every test conversation, reasoning effort included. Tool calls parse with the embedded template: 24 of 24 in two agentic coding runs on ik_llama.cpp cea2d61 with --jinja.
License
MIT, inherited from deepseek-ai/DeepSeek-V4-Flash-Vision-Exp. The uncensoring is orcarouter's work. Their card describes how it was done and the conditions they released it under.