Nanbeige4.3-33B GGUF

GGUF-quantized versions of Nanbeige/Nanbeige4.3-33B for local inference with llama.cpp.

What this is

Nanbeige4.3-33B is a 33B-parameter agentic teacher model from Nanbeige, built for complex coding and professional cowork tasks. It is a Llama-architecture model with 52 layers, 7168 hidden size, 56 attention heads (8 KV heads), 524288 context length, and a 166912-token vocabulary.

These are GGUF conversions of the original HF safetensors release — not a fine-tune or distillation of our own.

Available quantizations

| File | Size | Precision | ||---|---|---| || Nanbeige4.3-33B-q8_0.gguf | 35.2 GB | Q8_0 (8.5 BPW) | || Nanbeige4.3-33B-q6_k.gguf | 27.2 GB | Q6_K | || Nanbeige4.3-33B-q5_k_m.gguf | 23.5 GB | Q5_K_M (5.68 BPW) | || Nanbeige4.3-33B-q4_k_m.gguf | 20.1 GB | Q4_K_M (4.84 BPW) | || Nanbeige4.3-33B-q3_k_m.gguf | 16.3 GB | Q3_K_M (3.93 BPW) | || Nanbeige4.3-33B-q2_k.gguf | 12.7 GB | Q2_K (3.06 BPW) |

All six were converted from the same f16 GGUF base (converted from the HF safetensors), then quantized with llama.cpp. Quantize from f16, never from another quantization.

How to run

llama-cli -m Nanbeige4.3-33B-q6_k.gguf -p "Hello" -n 128

Chat template: the tokenizer supports enable_thinking (default on), preserve_thinking, and tool-use with tool_call_format="xml" (also json). Context length 512K.

Original model

• Repo: Nanbeige/Nanbeige4.3-33B
• License: Apache 2.0
• Parameters: 33.12B
• Tensor type: BF16
• Architecture: LlamaForCausalLM

Note

This is a conversion for local inference only. Benchmarks from the original release are reproduced here for reference; no local evaluation was run on these quantized files.