gpt-oss-120b-tq3a-tq2e-g32 (hybrid 3-bit attention / 2-bit experts)

TurboQuant hybrid quantization of openai/gpt-oss-120b using TurboQuant-MLX — 3-bit attention + 2-bit MoE experts at group size 32.

This is the smallest GPT-OSS-120B build that runs on a 48 GB Apple Silicon Mac while passing a full 6-test stress harness. ~35 GB on disk, ~38–39 GB peak wired RAM at decode, 42–50 tok/s.

Documentation: manjunathshiva.github.io/turboquant-mlx: the GPT-OSS page, getting started, choosing a model for your Mac's memory, guides for serving an OpenAI-compatible API, local coding agents and memory and macOS settings.

Why hybrid?

GPT-OSS-120B is a top-4 MoE: ~117 B total params but only ~5.1 B active per token. The MoE expert weights dominate disk size but are mostly idle during decode, so they tolerate aggressive quantization (2-bit). Attention is on the hot path every token, so it gets the higher-fidelity 3-bit treatment. The result is a model that fits 48 GB with comparable quality to full 3-bit at ~14 GB lower wired RAM.

Will it fit my Mac?

Don't guess, and don't download 37.7 GB to find out — ask first (needs turboquant-mlx-full >= 0.15.1):

turboquant-plan --model manjunathshiva/gpt-oss-120b-tq3a-tq2e-g32

It reads only this repo's safetensors headers over the network (a couple of hundred KB, a few seconds — the weights are never fetched), then projects the peak against your machine and prints the flags to use:

Model
  weights (exact)      37.7 GB
  KV cache             36.6 KB/token  (hybrid: 18/36 full-attention, 18 sliding (window 128))

Verdict: <resident | needs a wired-limit raise | streaming>
Recommended: <the exact flags for your Mac>

Planning for a machine you're not sitting at — deciding whether a 16 GB mini or a 64 GB Mac can take this — is what --wired-gb / --ram-gb are for:

turboquant-plan --model manjunathshiva/gpt-oss-120b-tq3a-tq2e-g32 \
    --wired-gb 10.5 --ram-gb 16 --context 8192

turboquant-doctor runs the same projection plus a readiness check; both take --json. The projection is calibrated against real measurements on a 16 GB Mac mini rather than estimated from theory — on the 9.4 GB ternary 35B it predicts a 10.44 GB peak where that machine measures 10.42.

Model Details

• Base Model: openai/gpt-oss-120b (top-4 MoE, ~117 B total / ~5.1 B active)
• Quantization: TurboQuant hybrid (Hadamard rotation + Lloyd-Max codebook)
  • Attention linears (q/k/v/o_proj): 3-bit
  • MoE expert linears (gate/up/down_proj): 2-bit
• Group size: 32
• Calibration data: none — TurboQuant is data-free
• Size: ~35 GB on disk
• Runs on: 48 GB Apple Silicon Mac (M-series) and up

Requirements

# macOS with Apple Silicon (M1/M2/M3/M4), 48 GB unified memory minimum
pip install turboquant-mlx-full mlx-lm

Quick Start

Download the model

huggingface-cli download manjunathshiva/gpt-oss-120b-tq3a-tq2e-g32 \
    --local-dir ~/models/gpt-oss-120b-tq3a-tq2e-g32

Generate text

turboquant-generate \
    --model ~/models/gpt-oss-120b-tq3a-tq2e-g32 \
    --prompt "Why is the sky blue? Give a thorough explanation." \
    --max-tokens 400 \
    --temp 0.7 --rep-penalty 1.1

Long contexts (>~2k tokens)

For prompts beyond ~2k tokens, enable KV cache compression to keep wired RAM under the 48 GB ceiling:

turboquant-generate \
    --model ~/models/gpt-oss-120b-tq3a-tq2e-g32 \
    --prompt "$(cat long_prompt.txt)" \
    --max-tokens 200 \
    --temp 0.7 --rep-penalty 1.1 \
    --kv-k-bits 8 --kv-v-bits 3 --kv-min-tokens 128

The KV flags compress past the first 128 tokens of context, keeping memory bounded on long inputs.

Verified quality (6-test stress harness)

All tests run on a 64 GB M-series Mac with --temp 0.7 --rep-penalty 1.1:

Decode speed across all 6 tests: 38–50 tok/s. Peak RAM: 38–39 GB.

How It Works

TurboQuant applies, in one shot with no calibration data:

1. Hadamard rotation — a reversible orthogonal transform that flattens weight outliers, so all values land in a narrow range that low-bit quantization can represent without large error.
2. Lloyd-Max codebook — optimal scalar centroids (4 levels for 2-bit, 8 for 3-bit) chosen to minimize quantization error against the rotated Gaussian-like distribution.
3. Group-wise scaling — per-group float16 scales (group size 32) preserve per-channel dynamic range.
4. Hybrid bit allocation — 3-bit on attention (every token), 2-bit on MoE experts (~5% active per token). Most of the size savings come from the experts; most of the quality preservation comes from attention.

The same recipe works for any standard transformer architecture without per-model calibration data.

License

Apache 2.0 (same as the base model).

Acknowledgements

• OpenAI — for releasing GPT-OSS-120B openly
• Google Research — for the original TurboQuant algorithm
• Apple — for MLX and the unified-memory architecture that makes this fit
• mlx-lm maintainers — for the GPT-OSS architecture support

Citation

@article{zandieh2025turboquant,
  title  = {TurboQuant: A Unified Framework for Extremely Low-Bit Weight and KV Cache Quantization},
  author = {Zandieh, Amir and Han, Minsik and Dalca, Andre and Shin, Jungwoo and Wang, Brian and Zhang, Yichao and Bordegoni, Matteo and Tian, Yuan and others},
  year   = {2025}
}

Repository

• TurboQuant-MLX (the conversion tool): https://github.com/manjunathshiva/turboquant-mlx
• Issues / questions: https://github.com/manjunathshiva/turboquant-mlx/issues