中文Sign up

burningfeet/backup2026-09-20-mudler-Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-GGUF

🤗 Hugging Face sourceapache-2.03B activated228 GBGGUF✓ 10 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo burningfeet/backup2026-09-20-mudler-Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-GGUF ./model-folder
Needs a seeder →

⚡ Each donation = another big MoE quantized

I host 25+ free APEX MoE quantizations as independent research. My only local hardware is an NVIDIA DGX Spark (122 GB unified memory), enough for ~30-50B-class MoEs, but bigger ones (200B+) require rented compute on H100/H200/Blackwell, typically $20-100 per quant.
If APEX quants are useful to you, your support directly funds those bigger runs.

🎉 Patreon (Monthly)  |  ☕ Buy Me a Coffee  |  ⭐ GitHub Sponsors

💚 Big thanks to Hugging Face for generously donating additional storage, much appreciated.

Qwen3.6 35B-A3B — Claude 4.7 Opus Reasoning Distilled — APEX GGUF

APEX (Adaptive Precision for EXpert Models) quantizations of lordx64/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled.

Brought to you by the LocalAI team | APEX Project | Technical Report

Available Files

File Profile Size Best For
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-I-Balanced.gguf I-Balanced 24 GB Best overall quality/size ratio
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-Balanced.gguf Balanced 24 GB General purpose
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-I-Quality.gguf I-Quality 21 GB Highest quality with imatrix
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-Quality.gguf Quality 21 GB Highest quality standard
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-I-Compact.gguf I-Compact 16 GB Consumer GPUs, best quality/size
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-Compact.gguf Compact 16 GB Consumer GPUs
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-I-Mini.gguf I-Mini 13 GB Smallest "safe" tier
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-I-Nano.gguf I-Nano 11 GB Experimental — IQ2_XXS mid-layer experts
Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-F16.gguf F16 reference 65 GB Full-precision reference
mmproj.gguf Vision projector ~1 GB Required for image understanding

What is APEX?

APEX is a quantization strategy for Mixture-of-Experts (MoE) models. It classifies tensors by role (routed expert, shared expert, attention) and applies a layer-wise precision gradient — edge layers get higher precision, middle layers get more aggressive compression. I-variants use diverse imatrix calibration (chat, code, reasoning, tool-calling, agentic traces, Wikipedia).

The key insight: in MoE models, expert FFN tensors make up the bulk of model weight but only ~8/256 experts activate per token. APEX compresses middle-layer experts more aggressively while preserving edge layers (first/last 5) and keeping attention, SSM/Mamba, and shared expert tensors at higher precision.

See the APEX project for full details, technical report, and scripts.

Nano (experimental tier)

The APEX Nano tier pushes mid-layer routed experts to IQ2_XXS (2.06 bpw), near-edge to IQ2_S, edges to Q3_K, with shared experts kept at Q5_K. About 20% smaller than Mini with modest quality cost — viable only on MoE thanks to sparse per-token expert activation. Requires imatrix.

Benchmarks pending. Feedback welcome.

Architecture

  • Model: Qwen3.6 35B-A3B Claude 4.7 Opus Reasoning Distilled
  • Base: Qwen 3.6 35B-A3B
  • Layers: 40
  • Experts: 256 routed + shared (8 active per token)
  • Total Parameters: ~35B
  • Active Parameters: ~3B per token
  • Attention: Hybrid (full attention every 4th layer, linear/Mamba otherwise)
  • Vision: Built-in vision encoder (mmproj included)
  • APEX Config: 5+5 symmetric edge gradient across 40 layers
  • Calibration: v1.3 diverse dataset (chat, code, reasoning, multilingual, tool-calling, Wikipedia)

Run with LocalAI

local-ai run mudler/Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-GGUF@Qwen3.6-35B-A3B-Claude-4.7-Opus-Reasoning-Distilled-APEX-I-Balanced.gguf

Credits