Laguna-XS.2 — TurboQuant 3-bit (MLX)

Data-free 3-bit quantization of poolside/Laguna-XS.2 for Apple Silicon via TurboQuant-MLX. Laguna XS.2 is a 33B-parameter Mixture-of-Experts (256 experts, top-8, 3B active) built by Poolside for agentic coding and long-horizon work on a local machine.

• 13.8 GB on disk, ~16 GB peak — runs comfortably on a 32 GB Mac (Poolside's own card quotes 36 GB for the bf16). Under Metal's ~75–81 % working-set limit, the practical floor is 32 GB with headroom.
• Data-free: no calibration set. Randomized Hadamard rotation → per-group RMS scale → Lloyd-Max codebook (optimal for the rotated Gaussian), decoded inline by fused Metal kernels.
• Agentic-capable: passes the Opencode observe → read → edit → verify loop.

Documentation: manjunathshiva.github.io/turboquant-mlx: the Poolside Laguna page, getting started, choosing a model for your Mac's memory, guides for serving an OpenAI-compatible API, local coding agents, models bigger than RAM and memory and macOS settings.

How to run

pip install "turboquant-mlx-full>=0.18.1"  # Laguna support; 0.17.0/0.18.0 are yanked
# generate
python -m turboquant_mlx.generate --model <this-repo> \
    --prompt "Write a mergesort in Python." --temp 0.7 --top-p 0.9
# OpenAI-compatible server (agentic use)
turboquant-serve --model <this-repo> --port 8080 \
    --temp 0.7 --top-p 0.9 --prompt-concurrency 1 \
    --chat-template-args '{"enable_thinking": false}'

Always pass --top-p 0.9. Laguna's generation_config ships top_p: 1.0; the untruncated 100k-vocab tail injects rare junk tokens even in otherwise clean output.

Do not pass --kv-bits. 30 of 40 layers are sliding-window(512), so KV stays tiny (peak 16.8 GB @2K → 19.4 GB @31K). KV-quant saves nothing here and can cost up to 4.4× decode.

The transformers tokenizer warning is harmless — ignore it. Recent transformers prints "incorrect regex pattern … this will lead to incorrect tokenization" and suggests fix_mistral_regex=True. Don't. This repo's tokenizer.json is structurally identical to the upstream Poolside release (same vocab, merges and pre-tokenizer), so the default is how the model was trained and what every other runtime uses. Setting the flag changes case-aware splitting (apoA, GeV, iPhone) on ~1% of real prompts and would make results diverge from every other Laguna deployment. All benchmarks below were run without it.

Will it fit my Mac?

Don't guess, and don't download 14.9 GB to find out — ask first (needs turboquant-mlx-full >= 0.18.1):

turboquant-plan --model manjunathshiva/Laguna-XS.2-tq3-g64

It reads only this repo's safetensors headers over the network (a couple of hundred KB, a few seconds — the weights are never fetched), then projects the peak against your machine and prints the flags to use. On a 64 GB M4 Max:

Model
  type                 laguna  (turboquant 3-bit g64)
  weights (exact)      14.85 GB

Machine
  Metal working set    55.66 GB   ← the real ceiling
  system RAM           68.72 GB

Projection at 16,384 tokens of context
  weights              14.85 GB
  KV cache             0.73 GB  (43.8 KB/token, hybrid: 10/40 full-attention, 30 sliding (window 512))
  prefill workspace    3.22 GB  (estimate, at --prefill-step-size 2048)
  runtime reserve      1.00 GB  (buffer cache, activations, fragmentation)
                       ----------------------------------
  peak                 19.81 GB of 61.85 GB usable   42.04 GB headroom

Verdict: ✅ RESIDENT — fits fully in memory

Planning for a machine you're not sitting at — deciding whether a 32 GB Mac can take this — is what --wired-gb / --ram-gb are for:

turboquant-plan --model manjunathshiva/Laguna-XS.2-tq3-g64 \
    --wired-gb 24 --ram-gb 32 --context 8192

Of the 14.85 GB, 13.62 GB is streamable experts and only 1.24 GB has to stay resident, so on a machine too small to hold it the planner recommends --streaming rather than giving up. (Laguna's expert container is recognised as streamable from turboquant-mlx-full >= 0.18.1; earlier releases counted it as resident and reported ❌ on small Macs.)

turboquant-doctor runs the same projection plus a readiness check; both take --json. The projection is calibrated against real measurements on a 16 GB Mac mini rather than estimated from theory — on the 9.4 GB ternary 35B it predicts a 10.44 GB peak where that machine measures 10.42.

Quality (6-prompt stress battery, temp 0.7 / top-p 0.9)

Standard accuracy (resident, no-think, temp 0.7; harness in scripts/benchmarks/):

Agentic coding (Opencode)

Passes the full agentic loop on the planted-bug fix task (run tests → locate bug → minimal edit → re-run → confirm). Serving notes:

• Serve with --top-p 0.9, --prompt-concurrency 1, enable_thinking:false.
• Tool calls use Laguna's native GLM-XML format; TurboQuant-MLX ships the parser fix for the trailing-newline function-name bug in mlx-lm's glm47 parser.

Speed & the 3-bit vs 4-bit tradeoff

Measured on an M4 Max (64 GB), resident, greedy. TurboQuant's data-free codebook 3-bit is smaller than MLX-native affine 4-bit but ~2× slower per token — because it trades bytes for compute: every forward pass decodes a Lloyd-Max codebook and applies an online Hadamard rotation, work that affine 4-bit (mx.quantized_matmul) does not. This is not a bandwidth effect — 3-bit reads fewer bytes; the cost is the extra per-token FLOPs, which is why the ~2× gap is uniform across prefill and decode:

Both pass Opencode with identical correctness. Pick this 3-bit build for the smallest download and lowest memory (32 GB Macs); pick affine 4-bit when you want the fastest interactive / agentic loop.

Peak memory vs context (this build):

16× the context costs only +2.6 GB — the sliding-window layout keeps KV nearly flat.

Provenance

• Base: poolside/Laguna-XS.2, released by Poolside under Apache-2.0; a verbatim copy of that licence ships here as LICENSE.md and applies to these derived weights. (Note for anyone comparing repos: Poolside's later Laguna S-2.1 is not Apache-2.0 — it is released under OpenMDW-1.1. The licence differs per model, so check per repo.)
• Converter: python -m turboquant_mlx.convert --hf-path poolside/Laguna-XS.2 --mlx-path out --bits 3 --group-size 64 --streaming.
• MLX Laguna support (model port + loader) is provided by TurboQuant-MLX's compat shim; mlx-lm has no native laguna model class as of 0.31.3.

Copyright 2026 Manjunath Janardhan. Model weights derive from poolside/Laguna-XS.2 and remain subject to the Apache-2.0 licence in LICENSE.md; Poolside's copyright and attribution notices are retained per that licence.