I host 30+ free APEX MoE quantizations as independent research. My only local hardware is an NVIDIA DGX Spark (122 GB unified memory), enough for ~30-50B-class MoEs, but bigger ones (200B+) require rented compute on H100/H200/Blackwell, typically $20-100 per quant.
If APEX quants are useful to you, your support directly funds those bigger runs.
🎉 Patreon (Monthly) | ☕ Buy Me a Coffee | ⭐ GitHub Sponsors
APEX (Adaptive Precision for EXpert Models) quantizations of huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated.
Brought to you by the LocalAI team | APEX Project | Technical Report
Status: unevaluated preview. No benchmarks were run on these files. The only check was loading each file and generating a short text with llama.cpp.
About the model. This is an abliterated (refusal-removed) fine-tune of a Claude-4.7-Opus-style reasoning distill of Qwen3.6-35B-A3B. Abliteration removes the model's built-in refusal behavior, so it will answer requests that the base model would decline. You are responsible for how you use it. These quants are derivatives of the base model and follow its license (Apache-2.0).
| File | Profile | Size | Best For |
|---|---|---|---|
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf | I-Balanced | 25.89 GB | General purpose, imatrix |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-Balanced.gguf | Balanced | 25.89 GB | General purpose |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Quality.gguf | I-Quality | 23.51 GB | Highest quality, imatrix |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-Quality.gguf | Quality | 23.51 GB | Highest quality |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Compact.gguf | I-Compact | 17.09 GB | Consumer GPUs, imatrix |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-Compact.gguf | Compact | 17.09 GB | Consumer GPUs |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Mini.gguf | I-Mini | 13.87 GB | Smallest tier, imatrix only |
| mmproj.gguf | Vision projector | 0.90 GB | Required for image understanding |
APEX is a quantization strategy for Mixture-of-Experts (MoE) models. It classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient: edge layers get higher precision, middle layers get more aggressive compression. I-variants use a diverse imatrix calibration (chat, code, reasoning, tool-calling, agentic traces, Wikipedia).
In MoE models the expert FFN tensors make up most of the weight, but only 8 of 256 experts activate per token. APEX compresses middle-layer experts more aggressively and keeps the edge layers, attention, SSM and shared-expert tensors at higher precision.
See the APEX project for details, the technical report and scripts.
mmproj.gguf)The source checkpoint carries unused multi-token-prediction tensors (blk.40.nextn.*). They are kept in the GGUF files. llama.cpp does not use them.
# text
llama-cli -m Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf -p "Your prompt" -ngl 99
# vision
llama-mtmd-cli -m Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf --mmproj mmproj.gguf -ngl 99
Needs a recent llama.cpp with qwen35moe support.
local-ai run mudler/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-GGUF@Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf
Apache-2.0, inherited from the base model Qwen3.6-35B-A3B. Keep the license and attribution when you redistribute these files.
Have the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo mudler/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-GGUF ./model-folder