Laguna-S-2.1-MLX-6bit

MLX 6-bit affine quantization of poolside/Laguna-S-2.1, packaged for Apple Silicon experiments and local OpenAI-compatible serving.

Laguna S 2.1 is Poolside's 118B-total / ~8B-active Mixture-of-Experts model for agentic coding and long-horizon software work. This repo keeps the same tokenizer, chat template, and Laguna custom code files, with the weights converted to MLX 6-bit group quantization.

This is a community conversion. Model architecture, license, intended use, and safety guidance belong to the original Poolside release.

Quick Facts

Why This Model Is Interesting

• Sparse-MoE design: 256 routed experts plus one shared expert, with top-10 routing.
• 48-layer Laguna S architecture with a 1:3 full-attention to sliding-window layout.
• 512-token sliding window layers help keep long-context decode memory bounded.
• Native reasoning controls through enable_thinking in the chat template.
• Built for coding, tools, and long-horizon agent workflows rather than short chat only.

Base Model Benchmarks

These are Poolside's reported base-model results for Laguna S 2.1, included here for context. They are not fresh benchmark runs of this MLX quantization.

See the original model card for the full table, comparison models, methodology notes, and links to trajectories.

Install

Laguna support in MLX is still moving quickly. If your installed mlx-lm cannot import model_type: laguna, use a Laguna-capable branch until support lands in your preferred release.

python -m venv ~/.venvs/mlx-laguna
source ~/.venvs/mlx-laguna/bin/activate


pip install -U mlx huggingface_hub
pip install -U "git+https://github.com/pierre427/mlx-lm.git@pr/laguna-windowed-kv-sanitize"

Generate With MLX

from mlx_lm import load, generate


model, tokenizer = load("TensorFold/Laguna-S-2.1-MLX-6bit", tokenizer_config={"trust_remote_code": True})


messages = [
    {"role": "user", "content": "Write a Python retry helper with exponential backoff."}
]


prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,
)


text = generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True)
print(text)

OpenAI-Compatible Server

This is the route I would use for local clients first. Keep concurrency conservative on a 256 GB Mac Studio, especially with long prompts.

source ~/.venvs/mlx-laguna/bin/activate


mlx_lm.server \
  --model TensorFold/Laguna-S-2.1-MLX-6bit \
  --host 0.0.0.0 \
  --port 8021 \
  --trust-remote-code \
  --chat-template-args '{"enable_thinking":false}' \
  --max-tokens 4096 \
  --decode-concurrency 1 \
  --prompt-concurrency 1 \
  --prefill-step-size 1024

Test it:

curl http://localhost:8021/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "default_model",
    "messages": [
      {"role": "user", "content": "Say hello in one short sentence."}
    ],
    "max_tokens": 64,
    "stream": false
  }'

Pi / OpenAI-Compatible Config

{
  "providers": {
    "MLX-Studio": {
      "baseUrl": "http://127.0.0.1:8021/v1",
      "api": "openai-completions",
      "models": [
        {
          "id": "default_model",
          "name": "Laguna-S-2.1-MLX-6bit",
          "reasoning": false,
          "input": ["text"],
          "contextWindow": 131072,
          "maxTokens": 4096
        }
      ]
    }
  }
}

Quantization Notes

• Converted with MLX 6-bit affine quantization.
• Group size is 64.
• MLX-LM recorded 6.501 bits per weight.
• The model config keeps MoE gate projections at 8-bit.
• Tokenizer, chat template, and Laguna remote-code files are included from the source model.
• Reasoning can be enabled, but many local clients behave better with enable_thinking=false unless they understand separate reasoning fields.

Known Caveats

• This is a large local model. Leave memory headroom for prompts, KV cache, and the OS.
• 6-bit is a middle ground between the smaller 4-bit repo and the higher-fidelity 8-bit repo.
• Some stable mlx-lm releases may not yet include Laguna support; a Laguna-capable branch may be required.
• DFlash speculative decoding for Laguna is not wired through stock MLX-LM at the time of this upload. Use Poolside's vLLM/SGLang/TRT recipes for the official DFlash serving path.
• The benchmark table above describes the base model, not a separate quantized eval run.

References

• Poolside Laguna S 2.1 model card
• Poolside release blog
• Laguna S 2.1 NVFP4 deployment notes
• MLX-LM

License

This quantized checkpoint follows the OpenMDW-1.1 license used by the original Poolside release.

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

Quick start and demo prompt

hf download TensorFold/Laguna-S-2.1-MLX-6bit --local-dir ./models/Laguna-S-2.1-MLX-6bit

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow TensorFold for new Apple Silicon releases and fixes.