SuperDeepseek V4 Flash IK GGUF

Mixed-precision GGUF quantizations of Jiunsong/SuperDeepseek-V4-Flash-abliterated, an abliterated derivative of DeepSeek V4 Flash 0731.

Compatibility warning

These files contain IK-specific IQK and IQT tensor types and require a compatible ik_llama.cpp build. Do not assume they will work in mainline llama.cpp, standard LM Studio, KoboldCpp, or another application using a different backend.

Build and testing used IK commit 3e416d7f5, build 4882. Set the context size explicitly. The model's advertised maximum context is not a practical default for most hardware.

Available files

Leave memory for the KV cache, runtime buffers, the operating system, and GPU offload granularity. File size alone does not establish whether a model fits.

Average bpw is an approximate, file-size-derived value relative to the 4.555-bpw MXFP4/BF16 intermediate. It includes non-expert tensors and GGUF overhead. IQ1_KT-based formats can be slower on CPU and CPU/GPU hybrid inference than their size alone suggests.

Perplexity measurements

Lower PPL is better on this corpus. Every row was measured from the release artifact before its NAS transfer, using llama-perplexity with:

• WikiText-2 raw test split, first 25 chunks;
• context 8192, batch and microbatch sizes 512;
• seed 1337, 16 threads;
• layer split 45,12, -ngl 999 --fit -mla 3 -amb 512;
• RTX 5090 and RTX 3080 hybrid offload on the same local machine.

These numbers are not directly comparable with results using another corpus, context, chunk selection, or runtime. Perplexity is not a roleplay-quality or refusal-rate benchmark.

IQ3_KS, IQ3_K, and IQ4_KSS exceed the available evaluation hardware limit. They passed file-layout, exact-template, and sampled finite-value checks, but were not inference-tested.

IQ2_KS stability check

The rest of the ladder was built only after IQ2_KS passed PPL and three seeded story probes through llama-server using the embedded Jinja template.

All three stopped without hitting the 8192-token generation limit. No repeated 12-word sequences were detected. Manual review found no runaway loops or catastrophic text degeneration, though there were ordinary writing and continuity flaws. The longest request fell short of its word target. These are limited smoke tests, not a guarantee for all prompts or long contexts.

Story settings were context 16384, temperature 0.9, top-p 0.95, repeat penalty 1.1, and thinking disabled. No new refusal-rate measurement was performed for this release.

Conversion and quantization

The source checkpoint was revision 93f0b5e3d1087d1e2ad12958fb89a394defe12e1. Conversion used mainline llama.cpp's HF-to-GGUF converter with --outtype bf16 --no-lazy --use-temp-file and a local precision override:

• all 129 routed expert tensors retained their native MXFP4 representation;
• FP8 paths were expanded to BF16 rather than Q8_0;
• the intermediate contained 555 BF16, 641 F32, and three I32 tensors alongside those experts.

IK quantization used --allow-requantize, mixed-precision recipes, and AtomicChat's original DeepSeek imatrix. Selected high-precision expert groups retain MXFP4. Reusing the parent-model imatrix is an approximation; it was not recalibrated on SuperDeepseek.

The exact corrected Unsloth Jinja template is embedded during native quantization. Tensor-region sizes, template identity, and sampled first/last tensor rows were checked for every file.

Running the model

For a compatible IK build:

./llama-server \
  -m SuperDeepseek-V4-Flash-abliterated-IQ2_KS.gguf \
  -c 8192 \
  -ngl 99 \
  --jinja

Adjust context and offload settings for your hardware. The embedded template supports the model's reasoning modes; the story smoke tests used thinking disabled.

Credits

• Abliterated checkpoint: Jiunsong/SuperDeepseek-V4-Flash-abliterated
• Original model: deepseek-ai/DeepSeek-V4-Flash-0731
• Quantization backend: ikawrakow/ik_llama.cpp
• Conversion: ggml-org/llama.cpp
• Importance matrix: AtomicChat/DeepSeek-V4-Flash-0731-GGUF
• Exact embedded chat template: unsloth/DeepSeek-V4-Flash-0731, the safetensors repository

License

Model weights follow the upstream MIT License. The included Unsloth chat template carries its own Apache-2.0 notice.