Qwen3.6-35B-A3B 3-bit MLX
3-bit quantized version of Qwen3.6-35B-A3B optimized for Apple Silicon inference via MLX.
Model Details
- Base Model: Qwen/Qwen3.6-35B-A3B
- Quantization: 3-bit (MLX format)
- Model Size: ~14 GB
- Architecture: Hybrid MoE (Mixture of Experts) + Mamba
- 10 full-attention layers
- 30 linear-attention (Mamba) layers — O(1) state per token
- Total Parameters: 35B (3B active per token)
- Context Length: 262,144 tokens (256K)
- Vocabulary: 248,320 tokens
Performance (M4 Pro 24GB)
| Metric | Value |
|---|---|
| Decode Speed | ~62 tok/s |
| Prefill Speed | ~400-540 tok/s |
| Max Context (24GB RAM) | 87,500 tokens |
| RAM Usage | ~15-16 GB |
Benchmark (Thinking OFF)
| Benchmark | Score |
|---|---|
| InstructFollow-15 | 75/100 |
| ReasonMath-15 | 72/100 |
| DataExtract-15 | 90/100 |
Comparison vs Q2_K_XL GGUF (same hardware)
| Metric | Q2_K_XL | 3-bit MLX |
|---|---|---|
| Decode Speed | 42 tok/s | 62 tok/s |
| Max Context | ~8K | 87.5K |
| Model Size | 11.4 GB | ~14 GB |
Usage
from mlx_lm import load, generate
model, tokenizer = load("andrevp/Qwen3.6-35B-A3B-3bit-MLX")
response = generate(model, tokenizer, prompt="Hello!", verbose=True)
Disable Thinking Mode
response = generate(
model, tokenizer,
prompt="Hello!",
max_tokens=512,
temp=0.0,
)
For chat template with thinking disabled, pass chat_template_kwargs={"enable_thinking": False} when applying the tokenizer.
Requirements
- Hardware: Apple Silicon Mac (M1/M2/M3/M4)
- RAM: 16 GB minimum, 24 GB recommended for long context
- Python: 3.11+
- MLX: 0.31+
- mlx-lm: latest
License
Apache 2.0 (same as base model)