Qwen3.5-122B-A10B-tq3a-tqTe-g64

Ternary (1.58-bit) expert TurboQuant quantization of Qwen/Qwen3.5-122B-A10B — 3-bit attention/dense, ternary {-c, 0, +c} experts packed as base-3 trits (group size 64) — produced with TurboQuant-MLX.

A 122B-parameter MoE compressed from ~240 GB (BF16) to 30.9 GB — a 7.8× reduction, and 43% smaller than the 3-bit build (54 GB) of the same model. Runs fully resident on 48 GB Macs (36 GB with a wired-limit bump), and streams its experts from SSD on machines as small as a 16 GB Mac mini.

Documentation: manjunathshiva.github.io/turboquant-mlx: the Qwen3.5-122B-A10B page, getting started, choosing a model for your Mac's memory, guides for serving an OpenAI-compatible API, local coding agents, models bigger than RAM and memory and macOS settings.

Model Details

Base model: Qwen/Qwen3.5-122B-A10B — Mixture-of-Experts (qwen3_5_moe, VLM; this build is used text-first)
Architecture: 48 layers, hidden size 3072, 256 routed experts (top-8), MoE intermediate 1024, ~122B total / ~10B active params
Quantization: TurboQuant (Hadamard rotation + Lloyd-Max codebook), group size 64
- Attention / dense → 3-bit
- Experts → ternary 1.58-bit (gate/up/down_proj of every routed expert): the data-free {-c, 0, +c} codebook, indices packed as genuine base-3 trits — 20 per uint32 = ~1.6 bpw
- Routers → full precision (auto-skipped)
Size: 30.90 GB (vs ~240 GB BF16; vs 54 GB for the tq3 sibling)

Why ternary works here

Ternary experts need redundancy to average out the coarse quantization, and this model has the best profile in the catalog: top-8 of 256 experts per token, quantized from a bf16 source. (For contrast: 32-expert models are word salad at sub-2-bit, and a 128-expert model quantized from an already-4-bit source degenerates too — expert count alone isn't the gate; it's experts × top-k routing × source precision.)

Note: this is a thinking-capable model. Use the chat template (the generator applies it automatically), and see the thinking-mode notes below.

Will it fit my Mac?

Don't guess, and don't download 30.9 GB to find out — ask first (needs turboquant-mlx-full >= 0.15.1):

turboquant-plan --model manjunathshiva/Qwen3.5-122B-A10B-tq3a-tqTe-g64

It reads only this repo's safetensors headers over the network (a couple of hundred KB, a few seconds — the weights are never fetched), then projects the peak against your machine and prints the flags to use:

Model
  weights (exact)      30.9 GB
    experts            27.0 GB  (streamable)
    resident           3.9 GB
  KV cache             24.0 KB/token  (hybrid: 12/48 full-attention)

Verdict: <resident | needs a wired-limit raise | streaming>
Recommended: <the exact flags for your Mac>

Planning for a machine you're not sitting at — deciding whether a 16 GB mini or a 64 GB Mac can take this — is what --wired-gb / --ram-gb are for:

turboquant-plan --model manjunathshiva/Qwen3.5-122B-A10B-tq3a-tqTe-g64 \
    --wired-gb 10.5 --ram-gb 16 --context 8192

turboquant-doctor runs the same projection plus a readiness check; both take --json. The projection is calibrated against real measurements on a 16 GB Mac mini rather than estimated from theory — on the 9.4 GB ternary 35B it predicts a 10.44 GB peak where that machine measures 10.42.

Quality

Validated with a 6-probe stress suite on a 64 GB Apple Silicon Mac (fully resident, temp 0.7). All six probes pass:

Probe	Result
1500-word essay (Roman Empire history)	✅ 12K chars, coherent, no degeneration
Multi-step math (two-trains meeting problem)	✅ Correct meeting time and distance
Code generation (`merge_intervals` + unit tests)	✅ Correct function with assert-based tests
Needle-in-haystack recall (password in ~4K-word filler)	✅ Returned the password exactly
Strict format (numbered list, per-line word limits)	✅ Perfect list under `--no-think` (see note)
Repetition trap (long open-ended explanation)	✅ No repetition collapse

Thinking-mode note: like other low-bit thinking builds, the deliberation phase is the fragile part — it can overrun tight max_tokens caps on strict-format tasks. As of turboquant-mlx-full 0.12.3 the mitigations ship as defaults: this repo's generation_config.json carries a light repetition_penalty (1.05) which the CLI applies automatically for thinking sessions, a second </think> is masked (kills doubled answers), and --no-think gives instant, loop-immune answers — recommended for math, strict formats, and latency-sensitive use.

Running it

Requires turboquant-mlx-full ≥ 0.12.3.

pip install "turboquant-mlx-full>=0.30.0" mlx-lm

Fully resident (48 GB Mac; on 36 GB first run sudo sysctl -w iogpu.wired_limit_mb=34816):

python -m turboquant_mlx.generate \
    --model manjunathshiva/Qwen3.5-122B-A10B-tq3a-tqTe-g64 \
    --prompt "Explain why the sky is blue." --max-tokens 512

On a 64 GB M4 Max (fully resident): ~25 tok/s* decode with 0.30.0 (measured ~13 tok/s on 0.29.0), 32.5 GB peak.

* Estimated for turboquant-mlx-full 0.30.0, not measured. 0.30.0 made ternary experts decode faster: the kernels no longer do an integer division per weight. The estimate comes from kernel timings at this model's expert shapes, calibrated against measured speeds; on NVIDIA Nemotron 3.5 Super VL it predicted 20.4 tok/s, and the build measured 21.0.

Expert streaming (16–32 GB machines — pages only the router-selected experts from SSD):

python -m turboquant_mlx.stream.stream_generate \
    --model manjunathshiva/Qwen3.5-122B-A10B-tq3a-tqTe-g64 \
    --prompt "Explain why the sky is blue." \
    --max-tokens 512 --cache-budget-gb 8

Measured on a 16 GB M4 Mac mini (internal SSD, --cache-budget-gb 8): ~2.1 tok/s end-to-end, 11.3 GB peak RSS, 82% expert-cache hit rate. Adding --max-active-experts 3 reached ~2.4 tok/s with quality intact; pushing K lower gains little — the bottleneck is per-layer read latency across 48 MoE layers, not bandwidth. Set expectations accordingly: on 16 GB this model works, producing coherent full-quality answers, but a thinking-mode answer takes several minutes. Notes: --cache-budget-gb 8 is the ceiling on 16 GB (larger budgets exhaust RAM), and streaming this build reads ~43% fewer bytes per token than the 54 GB tq3 sibling. For an interactive 16 GB experience, use the 35B ternary sibling (fully resident, ~10 tok/s) — this 122B earns its keep fully resident on 36 GB+ Macs.

For an OpenAI-compatible server: pip install "turboquant-mlx-full>=0.30.0" then turboquant-serve --model manjunathshiva/Qwen3.5-122B-A10B-tq3a-tqTe-g64 --kv-bits 8 (add --cache-budget-gb 4 on small machines; --chat-template-args '{"enable_thinking": false}' to serve with thinking off).

Choosing a build

Build	Experts	Size	Fits resident	Decode (64 GB M4 Max, resident)	Notes
tq3a-tqTe-g64 (this)	ternary ~1.6 bpw	30.9 GB	36–64 GB, no sysctl needed on 48+	~25 tok/s* (0.29.0: ~13)	smallest; long-context headroom; fastest streaming on minis
tq3-g32	3-bit	54 GB	64 GB only, sysctl required	~25 tok/s (but Metal-OOMs past ~4–8K context)	recall-critical work

The trade in one line: through 0.29.0, ternary halved the decode speed but halved the footprint, because base-3 trit decode cost more GPU ALU than 3-bit shift-and-mask. 0.30.0 removed most of that cost, so this build should now decode at about the 3-bit build's speed* in about half the memory. Pick tq3 for recall-critical work; pick this build for smaller Macs, zero setup, and room for long contexts.

How it was made

python -m turboquant_mlx.convert \
    --hf-path Qwen/Qwen3.5-122B-A10B \
    --mlx-path Qwen3.5-122B-A10B-tq3a-tqTe-g64 \
    --bits 3 --group-size 64 --ternary-experts

Data-free (no calibration set): randomized Hadamard rotation → per-group RMS scaling → Lloyd-Max codebook (ternary for experts) → base-3 trit packing. The 3-entry codebook is self-describing on disk — loaders and Metal kernels switch to base-3 decode automatically.

manjunathshiva/Qwen3.5-122B-A10B-tq3a-tqTe-g64