⚡ Each donation = another big MoE quantized
I host 30+ free APEX MoE quantizations as independent research. My only local hardware is an NVIDIA DGX Spark (122 GB unified memory), enough for ~30-50B-class MoEs, but bigger ones (200B+) require rented compute on H100/H200/Blackwell, typically $20-100 per quant.
If APEX quants are useful to you, your support directly funds those bigger runs.
🎉 Patreon (Monthly) | ☕ Buy Me a Coffee | ⭐ GitHub Sponsors
Huihui-Qwen3.6-35B-A3B Claude 4.7 Opus abliterated - APEX GGUF
APEX (Adaptive Precision for EXpert Models) quantizations of huihui-ai/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated.
Brought to you by the LocalAI team | APEX Project | Technical Report
Status: unevaluated preview. No benchmarks were run on these files. The only check was loading each file and generating a short text with llama.cpp.
About the model. This is an abliterated (refusal-removed) fine-tune of a Claude-4.7-Opus-style reasoning distill of Qwen3.6-35B-A3B. Abliteration removes the model's built-in refusal behavior, so it will answer requests that the base model would decline. You are responsible for how you use it. These quants are derivatives of the base model and follow its license (Apache-2.0).
Available Files
| File | Profile | Size | Best For |
|---|---|---|---|
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf | I-Balanced | 25.89 GB | General purpose, imatrix |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-Balanced.gguf | Balanced | 25.89 GB | General purpose |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Quality.gguf | I-Quality | 23.51 GB | Highest quality, imatrix |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-Quality.gguf | Quality | 23.51 GB | Highest quality |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Compact.gguf | I-Compact | 17.09 GB | Consumer GPUs, imatrix |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-Compact.gguf | Compact | 17.09 GB | Consumer GPUs |
| Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Mini.gguf | I-Mini | 13.87 GB | Smallest tier, imatrix only |
| mmproj.gguf | Vision projector | 0.90 GB | Required for image understanding |
What is APEX?
APEX is a quantization strategy for Mixture-of-Experts (MoE) models. It classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient: edge layers get higher precision, middle layers get more aggressive compression. I-variants use a diverse imatrix calibration (chat, code, reasoning, tool-calling, agentic traces, Wikipedia).
In MoE models the expert FFN tensors make up most of the weight, but only 8 of 256 experts activate per token. APEX compresses middle-layer experts more aggressively and keeps the edge layers, attention, SSM and shared-expert tensors at higher precision.
See the APEX project for details, the technical report and scripts.
The model
- Model: Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated by huihui-ai
- Base: Qwen3.6-35B-A3B
- Layers: 40
- Experts: 256 routed plus a shared expert, 8 active per token
- Parameters: about 35B total, about 3B active per token
- Attention: hybrid, full attention every 4th layer and linear attention otherwise
- Vision: built-in vision tower (use
mmproj.gguf) - Calibration: v1.3 diverse dataset for the I-variants
The source checkpoint carries unused multi-token-prediction tensors (blk.40.nextn.*). They are kept in the GGUF files. llama.cpp does not use them.
Usage
# text
llama-cli -m Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf -p "Your prompt" -ngl 99
# vision
llama-mtmd-cli -m Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf --mmproj mmproj.gguf -ngl 99
Needs a recent llama.cpp with qwen35moe support.
Run with LocalAI
local-ai run mudler/Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-GGUF@Huihui-Qwen3.6-35B-A3B-Claude-4.7-Opus-abliterated-APEX-I-Balanced.gguf
License
Apache-2.0, inherited from the base model Qwen3.6-35B-A3B. Keep the license and attribution when you redistribute these files.