Qwen3.6-35B-A3B-MLX-NVFP4
Summary
MLX-NVFP4 (4-bit) variant of Qwen/Qwen3.6-35B-A3B.
Why this variant
Apple Silicon (M1/M2/M3/M4) with the NVIDIA Blackwell-native FP4 layout (E2M1 weights, per-16-element FP8 scales) — reproduced on-device via MLX. ~4.5 bits/weight, ~18 GB on disk, loads in under 2 s on M4 Max. Pick this for NVFP4-format parity with Blackwell pipelines while running locally.
Quickstart
import mlx.core as mx
from mlx_lm import load, generate
model, tokenizer = load("majentik/Qwen3.6-35B-A3B-MLX-NVFP4")
prompt = "Give me a short introduction to Mixture-of-Experts models."
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)
print(response)
[!IMPORTANT] Text-only extraction. This repo contains only the quantized language tower of Qwen3.6-35B-A3B. The upstream vision tower (333 tensors) and MTP head are not included, so image/video input does not work and
mlx_vlm.load(...)fails with aMissing ... parameterserror (the vision tower it expects is absent from the checkpoint). Load it withmlx_lm(recent version withqwen3_5_moesupport) as shown above. For image/video input, use the upstream BF16 model Qwen/Qwen3.6-35B-A3B on a runtime that supports it.
Hardware compatibility
| Device | VRAM | Recommendation |
|---|---|---|
| Apple M4 Max 128 GB | ~21 GB | recommended — headroom for long context |
| Apple M3 Max 64 GB | ~21 GB | fits comfortably |
| Apple M2 Max 32 GB | ~21 GB | tight — short context only |
Reproduce
python -c "from mlx_lm import convert; convert(hf_path=\"Qwen/Qwen3.6-35B-A3B\", mlx_path=\"out\", quantize=True, q_bits=4, q_group_size=16, q_mode=\"nvfp4\", trust_remote_code=True)"
Reproduced at commit 919836a.
Evaluation
| Benchmark | Score |
|---|---|
| arc_easy_acc | 0.5150 |
| hellaswag_acc | 0.5000 |
Family
- bf16 — Qwen/Qwen3.6-35B-A3B
- FP8 (official) — Qwen/Qwen3.6-35B-A3B-FP8
- FP8 card — majentik/Qwen3.6-35B-A3B-FP8
- MLX-MXFP4 (this sibling) — majentik/Qwen3.6-35B-A3B-MLX-MXFP4
Provenance
- Source SHA:
upstream-Qwen3.6-35B-A3B-bf16 - Calibration hash:
none (nvfp4 is calibration-free) - Uploaded:
2026-04-21T04:08:52.357536+00:00
Toolchain:
huggingface_hub: 1.11.0mlx: 0.31.1mlx-lm: 0.31.2
License
Released under apache-2.0. Upstream license of the base model applies.