Nanbeige4.3-33B-oQ8e

This repository holds a complete Apple MLX build of Nanbeige4.3-33B, quantized with oMLX's oQ method at a nominal 8 bits.

Nanbeige4.3-33B is the Nanbeige team's flagship of the 4.3 series: a dense 33B model for agentic coding and office work ("cowork"), with a reasoning mode and tool calling. The team built it as the teacher for smaller models of the series. The source model card has its benchmark results.

Download the whole repository: the Safetensors shards need the included index, model config, tokenizer and chat template. This is not a GGUF or Transformers artifact.

Quick facts

Choose a variant

No quality, memory, time-to-first-token or throughput comparison on one Mac is published here; the table compares sizes only.

Memory for long contexts

Besides the weights, the KV cache needs memory, and it grows with the context. At fp16 it takes 208 KiB per token (52 layers × 8 KV heads × 128 dimensions × keys and values × 2 bytes) in all three variants:

The numbers are estimates from the config for one sequence without KV-cache quantization. The runtime adds its own buffers.

Download

Install the Hugging Face CLI, then put the complete repository below oMLX's model directory:

mkdir -p "$HOME/.omlx/models/pyros-vault"
hf download pyros-vault/Nanbeige4.3-33B-oQ8e \
  --local-dir "$HOME/.omlx/models/pyros-vault/Nanbeige4.3-33B-oQ8e"

Serve with oMLX

Install the current oMLX runtime and start its OpenAI-compatible server:

brew tap jundot/omlx https://github.com/jundot/omlx
brew install jundot/omlx/omlx
omlx serve --model-dir "$HOME/.omlx/models"

Look up the model ID your oMLX version exposes:

curl http://127.0.0.1:8000/v1/models

Then use that ID with the OpenAI-compatible endpoint.

Chat template and sampling

The bundled chat template is the upstream one. The Nanbeige team describes its arguments:

• enable_thinking turns the reasoning on (the default); set it to false for answers without reasoning.
• preserve_thinking keeps the reasoning of earlier assistant turns. The team recommends false for chat and questions, and true for multi-turn tool use, office tasks and coding agents.
• With tools, the template switches to tool use; tool_call_format="xml" gives the best tool calls, json also works.

Whether a request can set these arguments depends on your oMLX version and client.

Upstream's sampling recommendations:

Quantization and verification

The bundled oq_imatrix_report.json records calibration with oqe_code_multilingual over 128 sequences of 512 tokens. It lists 364 importance entries applied to 364 linear modules, one missing name (lm_head), and no shape mismatches. All three variants use the same importance matrix. It was computed for the oQ4e build; this report records it as reused.

This card rests on the repository's structure, configs, shard count, payload size and quantization metadata. It makes no claim about speed, memory use, long-context behavior or output quality on a Mac.

Limitations

• MLX/oMLX targets Apple Silicon and macOS; this repository does not run through CUDA.
• Hugging Face hosted inference does not serve this oMLX layout.
• The 524,288-token context is the config's value, not a claim that this context was tested or fits your machine (see the KV-cache table above).
• Reasoning, tool calls, memory use and throughput depend on the oMLX version, the client, the prompt and the Apple hardware.
• Quantization can change output quality. Evaluate this variant on your own workload.

License and credits

The Nanbeige team released Nanbeige4.3-33B under Apache-2.0; this quantization keeps that license. For questions about the model itself, see the source model card, and cite it as the Nanbeige team asks there.

Quantized and packaged by pyros-vault with oMLX/oQ.