Nanbeige-4.3-33B-Heretic-GGUF
Quantized GGUF weights for Luni/Nanbeige-4.3-33B-Heretic, an abliterated and de-slopped dense 33B model derived from Nanbeige/Nanbeige4.3-33B.
Highlights
- Base model architecture: 52-layer pure dense causal LM with full attention and native
<think>chain-of-thought reasoning. - Refusal alignment removed via Heretic directional ablation while retaining mathematical, algorithmic, and analytical reasoning depth.
- All IQ (Importance Matrix) quants calibrated using a specialized corpus of high-IQ STEM anchors, lock-free systems programming, distributed systems, and competitive math (
nanbeige_heretic.imatrix, PPL: 26.39). - Upstream double
<|im_start|>BOS token issue patched (add_bos_token: false).
Available Quantizations
| File | Type | BPW | Size (GB) | Recommended VRAM / RAM | Description |
|---|---|---|---|---|---|
Nanbeige-4.3-33B-Heretic.Q8_0.gguf |
Q8_0 | 8.00 | ~35.0 | 48 GB+ | Near-lossless reference quality |
Nanbeige-4.3-33B-Heretic.Q6_K.gguf |
Q6_K | 6.56 | ~28.5 | 36 GB+ | High-precision flagship |
Nanbeige-4.3-33B-Heretic.Q5_K_M.gguf |
Q5_K_M | 5.54 | ~24.5 | 32 GB | Balanced high quality |
Nanbeige-4.3-33B-Heretic.Q5_K_S.gguf |
Q5_K_S | 5.34 | ~23.8 | 32 GB | Compact 5-bit |
Nanbeige-4.3-33B-Heretic.Q4_K_M.gguf |
Q4_K_M | 4.58 | ~20.5 | 24 GB / 32 GB | Standard generalist 4-bit |
Nanbeige-4.3-33B-Heretic.Q4_K_S.gguf |
Q4_K_S | 4.37 | ~19.8 | 24 GB | Compact 4-bit |
Nanbeige-4.3-33B-Heretic.IQ4_NL.gguf |
IQ4_NL | 4.50 | ~19.0 | 24 GB | Non-linear 4-bit imatrix (recommended) |
Nanbeige-4.3-33B-Heretic.IQ4_XS.gguf |
IQ4_XS | 4.25 | ~19.2 | 24 GB | Extra-small 4-bit imatrix |
Nanbeige-4.3-33B-Heretic.Q3_K_L.gguf |
Q3_K_L | 3.81 | ~18.0 | 20 GB / 24 GB | Large 3-bit |
Nanbeige-4.3-33B-Heretic.Q3_K_M.gguf |
Q3_K_M | 3.52 | ~17.2 | 20 GB | Medium 3-bit |
Nanbeige-4.3-33B-Heretic.Q3_K_S.gguf |
Q3_K_S | 3.20 | ~15.9 | 16 GB / 20 GB | Small 3-bit |
Nanbeige-4.3-33B-Heretic.IQ3_M.gguf |
IQ3_M | 3.70 | ~17.0 | 20 GB | Medium 3-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ3_S.gguf |
IQ3_S | 3.44 | ~16.0 | 16 GB | Small 3-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ3_XS.gguf |
IQ3_XS | 3.30 | ~15.2 | 16 GB | Extra-small 3-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ3_XXS.gguf |
IQ3_XXS | 3.06 | ~14.2 | 16 GB | Minimum 3-bit imatrix |
Nanbeige-4.3-33B-Heretic.Q2_K.gguf |
Q2_K | 2.63 | ~13.8 | 16 GB | Standard 2-bit |
Nanbeige-4.3-33B-Heretic.IQ2_M.gguf |
IQ2_M | 2.70 | ~12.8 | 16 GB | Medium 2-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ2_S.gguf |
IQ2_S | 2.50 | ~12.0 | 12 GB / 16 GB | Small 2-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ2_XS.gguf |
IQ2_XS | 2.31 | ~11.2 | 12 GB | Extra-small 2-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ2_XXS.gguf |
IQ2_XXS | 2.06 | ~10.2 | 12 GB | Minimum 2-bit imatrix |
Nanbeige-4.3-33B-Heretic.IQ1_M.gguf |
IQ1_M | 1.75 | ~8.8 | 8 GB / 12 GB | Extreme 1.75 bpw compression |
Nanbeige-4.3-33B-Heretic.IQ1_S.gguf |
IQ1_S | 1.56 | ~8.0 | 8 GB | Extreme 1.56 bpw compression |
Usage with llama.cpp
# Run with CLI
llama-cli -m Nanbeige-4.3-33B-Heretic.IQ4_NL.gguf -p "<|im_start|>user\nSolve for x: 3x + 7 = 22<|im_end|>\n<|im_start|>assistant\n<think>\n" -ngl 99 -c 32768
# Run with Server
llama-server -m Nanbeige-4.3-33B-Heretic.IQ4_NL.gguf --port 8080 -ngl 99 -c 32768
Model Lineage & Related Repositories
- Base Model: Nanbeige/Nanbeige4.3-33B by Nanbeige Team (Apache-2.0)
- Heretic BF16: Luni/Nanbeige-4.3-33B-Heretic
- Stock EXL3: Luni/Nanbeige-4.3-33B-EXL3
- Heretic EXL3 Master Suite: Luni/Nanbeige-4.3-33B-Heretic-EXL3
- Dedicated SC Repo: Luni/Nanbeige-4.3-33B-Heretic-EXL3-SC