Support on Ko-fi or X Money: @DJLougen

Nanbeige4.3-33B-MLX

Nanbeige4.3-33B quantized for MLX: 5 variants, 13.1–66.2 GB (3.16–16.00 bits per weight). The 4bit runs at 13.9 tok/s on a 36 GB MacBook Pro (measured peak 17.5 GiB)

MLX conversions of Nanbeige/Nanbeige4.3-33B (revision c1093c5, a 33B dense Llama-architecture reasoning model, 52 layers, 8 KV heads) made with mlx-lm 0.31.3. Each variant ships in its own subfolder.

Downloads

Official source: Nanbeige/Nanbeige4.3-33B. Please credit Nanbeige for the original model. If you share these conversions, please link here rather than re-uploading.

At a glance

Peak is the measured MLX peak for a 256-token greedy generation on a DGX Spark (mlx CUDA backend), so it isn't a device-fit claim. Judgement labels are computed from the measured top-1 agreement: PERFECT ≥ 95%, GREAT ≥ 80%, GOOD ≥ 50%, MISS = below that or a failed smoke test. mixed_2_6 passes the one-line smoke test, but its KL is 6.7× that of 4bit. Treat it as a degraded option for when 4bit doesn't fit.

Quick start

pip install mlx-lm
hf download DJLougen/Nanbeige4.3-33B-MLX --include "4bit/*" --local-dir nanbeige-mlx
mlx_lm.generate --model nanbeige-mlx/4bit --prompt "Explain what a QR code encodes." --max-tokens 512

Nanbeige4.3 is a reasoning model and thinks by default (<think> … </think> before the answer). To answer directly, add --chat-template-config '{"enable_thinking": false}'. Swap 4bit for any folder in the table. The subfolder name is the variant.

4bit on a 36 GB MacBook Pro (long prompts)

Single stream, bf16 KV cache, prefill chunk 512, greedy, 128 new tokens. Each figure is the median of 2–3 runs. The files are byte-identical to 4bit/ (SHA-256 checked).

The model's config allows 524,288 positions. On a 36 GB laptop with normal apps open, ~16K tokens was the practical ceiling before the system ran short of memory. KV cache costs ~200 KiB/token in bf16. 8-bit KV (kv_bits=8) did not lower peak memory here: at 16K it was 21.16 vs 20.98 GiB, it was slower, and so it isn't recommended.

Exact numbers (KL vs MLX bf16, 8,192 tokens, 16 seqs × 512 tokens, DGX Spark, mlx 0.32.4)

Peak and Gen tok/s here come from the DGX Spark (GB10, mlx CUDA backend), not a Mac: a 256-token greedy chat generation, wall-clock including prefill, with the GPU otherwise idle. Text source: deterministic 60/20/20 interleave of: wikitext-2-raw-v1 test split (Salesforce/wikitext) for prose; CPython 3.12.3 stdlib argparse.py for code; openai/gsm8k main train split (question+answer text, first 400) for math. Packed 16x512 tokens, each sequence starting with BOS. Source stated for reproducibility; NOT claimed held-out. Logits forward: chat-template-free raw text, BOS + 511 text tokens per sequence. The bf16 row is the A/A control (variant = reference, so it must be exactly 0 / 100%).

How it was made

Converted from the verified upstream checkpoint (all 14 shards matched the Hub's SHA-256 at revision c1093c5) with mlx-lm 0.31.3:

mlx_lm.convert --hf-path Nanbeige/Nanbeige4.3-33B --mlx-path bf16 --dtype bfloat16
mlx_lm.convert --hf-path Nanbeige/Nanbeige4.3-33B --mlx-path 8bit -q --q-bits 8 --q-group-size 64   # same for 6 / 4
mlx_lm.convert --hf-path Nanbeige/Nanbeige4.3-33B --mlx-path mixed_2_6 -q --q-group-size 64 --quant-predicate mixed_2_6

mixed_2_6 is mlx-lm's stock mixed recipe, read from its config: 2-bit (group 64) for q/k/o/gate/up projections and the token embeddings, and 6-bit for lm_head and for v_proj / down_proj in 26 of the 52 layers (313 × 2-bit + 53 × 6-bit tensors, 3.16 bpw overall). It's the smallest variant that still passes the smoke test, but it is much further from bf16 than 4bit (58% vs 83% top-1). Use it only if 4bit doesn't fit.

Recipes we measured and didn't ship

Click to expand

Limitations

• No task accuracy suite was run (no MMLU, GSM8K or other benchmark scores). Quality is proxied by KL divergence and top-1 agreement against the MLX bf16 conversion, plus a smoke test.
• One machine per measurement type, one text set, one seed. The bf16 MLX conversion was not compared against upstream HF transformers.
• bf16, 8bit and 6bit were not run on a Mac. Their peak memory and speed in the detail table are DGX Spark numbers.
• Reasoning/thinking quality and long-context retrieval were not evaluated per variant.

Verification

SHA256SUMS lists every weight file. The scripts and raw outputs behind every number are in results/: kl_eval.py, kl_results.json, mac_eval.py, mac_eval.jsonl, mac_4bit_ctx_sweep.json, the build chain, and this card's generator build_card.py.

License & credits

Released under Apache-2.0, inherited from the upstream model. All credit for the model goes to Nanbeige (Nanbeige/Nanbeige4.3-33B).

Support on Ko-fi or X Money: @DJLougen