CyberNeurova DeepSeek V4 Flash Abliterated v2 — IQ3_XXS-AS F32Router GGUF

Tailor-made for the 128 GB M5 Max MacBook Pro. This quant was designed, built, and benchmarked on that exact Apple-silicon machine. Its 95.48 GiB weights fit inside the tested Mac's 107.5 GiB Metal-visible allocation pool while leaving about 12 GiB for runtime state and context.

This is a custom importance-matrix quantization of cyberneurova/CyberNeurova-DeepSeek-V4-Flash-abliterated-GGUF. It specifically targets the 128 GB unified-memory M5 Max MacBook Pro and similar machines where the Q8 release is too large but preserving more quality than the smallest quants is worthwhile.

This repository only contributes the quantization. CyberNeurova created the abliterated v2 weights; DeepSeek created the original V4 Flash model.

Artifact

llama-gguf-hash tensor-data SHA-256: 12e66621bd390b79904463cf3cdb5b288bf7265352cf1b66740e211c8f39c7b3.

Quantization layout

The installed llama-quantize 10090 dry run reported:

model size = 288244.36 MiB (8.50 BPW)
quant size =  97767.25 MiB (2.88 BPW)
imatrix entries = 129

The tensor layout was reconstructed by comparing the CyberNeurova Q8 GGUF with bullerwins/DeepSeek-V4-Flash.IQ3_XXS-AS.gguf. Both contain the same 1,328 tensor names and dimensions. The published AS reference stores its 43 router-input tensors as BF16. Current llama.cpp excludes those tensors from re-quantization, so this build preserves CyberNeurova's F32 routers instead. That costs about 86 MiB and retains higher precision.

The complete per-tensor mapping is included at recipe/iq3_xxs_as.tensor-types.txt.

Inputs and reproducibility

The importance matrix is the published Jedi chat-v2 routed-MoE matrix used for Audrey Tang's aligned CyberNeurova re-quant.

The build used llama.cpp version 10090 (7347430f4) and:

llama-quantize \
  --allow-requantize \
  --imatrix DeepSeek-V4-Flash-chat-v2-routed-moe-ds4-aligned.dat \
  --tensor-type-file recipe/iq3_xxs_as.tensor-types.txt \
  --override-kv \
  general.name=str:CyberNeurova-DeepSeek-V4-Flash-Abliterated-v2-IQ3_XXS-AS-F32Router \
  cyberneurova-DeepSeek-V4-Flash-abliterated-Q8_0.gguf \
  CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-F32Router.gguf \
  IQ3_XXS

Validation checked every tensor name, dimension, and requested tensor type. It also hash-compared every F32, I32, and Q8_0 tensor expected to remain byte-identical to the CyberNeurova Q8 source.

Download

hf download \
  pyrodog/CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-GGUF \
  CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-F32Router.gguf \
  chat_template.jinja \
  --local-dir .

Run with llama.cpp

The GGUF does not embed a chat template. The companion chat_template.jinja is the exact template extracted from a clean DeepSeek V4 Flash reference GGUF. Pass it explicitly; otherwise llama.cpp may fall back to ChatML and produce misleadingly poor behavior.

The following is the tested 32K-context Apple Metal profile:

llama-server \
  -m CyberNeurova-DeepSeek-V4-Flash-abliterated-v2-IQ3_XXS-AS-F32Router.gguf \
  -ngl 999 \
  -fa on \
  -ctk q8_0 \
  -ctv q8_0 \
  -c 32768 \
  --jinja \
  --chat-template-file chat_template.jinja \
  --reasoning-format deepseek \
  --reasoning on

The model was tested on a 128 GB M5 Max. The 95.48 GiB weight file left about 12 GiB of that machine's 107.5 GiB Metal-visible allocation pool for runtime state and context. Actual usable memory varies by system and background load.

Local A/B measurements

The custom quant was compared with Audrey Tang's 90.89 GiB mixed quant under the same mainline llama.cpp Metal runtime. Both used all GPU layers, flash attention, Q8 KV caches, a 32,768-token context, temperature 0, and seed 424242.

For 4.59 GiB more storage, this quant was 42.05% faster on the 512-token prompt, 10.41% faster on the 8,192-token prompt, and 12.09% faster in generation. Its measured perplexity was 9.90% lower.

These are controlled measurements on one machine, not broad independent benchmarks. The 12-task smoke suite is particularly small; a formatting-only miss on each side makes the semantically lenient score a 7/12 tie. The speed and perplexity results are the stronger reasons to prefer this quant.

Audrey's specialized ds4 runtime can execute the smaller mixed quant's IQ2/Q4 experts much faster, but does not implement this model's IQ3 expert dispatch. Use this quant for the measured fidelity/performance balance under mainline llama.cpp; use the smaller mixed quant with ds4 when specialized decode speed is the priority.

Intended use and limitations

This is an experimental research artifact for local inference, defensive security research, academic study of refusal mechanisms, red-team evaluation, and controlled coding-agent experiments.

The underlying model has had canonical refusal behavior deliberately reduced. That does not make its outputs correct or safe. It can produce harmful, insecure, biased, or fabricated content. Review generated code before running it, use sandboxing for tools, and do not use the model to facilitate illegal activity or harm.

Additional limitations:

• Long-context behavior beyond 32K was not validated for this quant.
• The benchmark covers one Apple-silicon machine and one llama.cpp build.
• Quantization adds error on top of the abliterated source weights.
• The GGUF requires the external chat template included in this repository.
• Mainline and specialized DeepSeek V4 runtimes are evolving quickly.

License and acknowledgements

MIT, inherited from DeepSeek V4 Flash and the CyberNeurova release. See LICENSE.

Thanks to:

• DeepSeek-AI for V4 Flash
• CyberNeurova for the v2 abliterated weights
• Audrey Tang for the aligned comparison quant and runtime work
• Jedi for the aligned importance matrix
• bullerwins for the IQ3_XXS-AS tensor-layout reference
• llama.cpp contributors for GGUF tooling and inference support