Nanbeige-4.3-33B-Heretic-EXL3-SC
Dedicated Self-Calibrated (SC) ExLlamaV3 quants for Luni/Nanbeige-4.3-33B-Heretic.
Released Branches
| Branch | Target VRAM | Size | Description |
|---|---|---|---|
3.50bpw |
16 GB VRAM | 14.0 GB |
Near-lossless coding and math reasoning (KLD ~0.035) |
3.00bpw |
16 GB VRAM | 12.0 GB |
Solid reasoning preservation with headroom for 32k context |
2.50bpw |
12 GB - 16 GB VRAM | 10.0 GB |
High-efficiency 12GB sweet spot with native 2.5 bpw trellis codebook |
2.25bpw |
12 GB VRAM | 9.4 GB |
Optimized for RTX 3060 12GB / 4070 12GB with 8k-16k context |
Optimization Pipeline
This model suite was quantized using turboderp's full 5-step Self-Calibrated (SC) optimization pipeline:
- Self-Sampled Trace (
sc_trace.py): Generated 571,666 in-domain<think>tokens across 140 chat conversations with internal reasoning enabled, packed intocal_trace.safetensors(250 x 2048). - Error Anchor Probe (
sc_rfn_probe.py): Measured relative Frobenius quantization errors across 468 linear tensors against an anchor quant. - LDLQ-Shaped Noise Attribution (
sc_measure.py --shaped): Evaluated output KL divergence degradation under shaped noise injection directly on the unquantized BF16 model. - Per-Tensor Integer Programming (
sc_optimize.py): Solved optimal per-tensor bit allocations, prioritizing critical mid-stack attention projections (q_proj,k_proj,v_proj) while squeezing tolerant MLP layers (up_proj,gate_proj). Gate projections are clamped to >= 4.0 bits andlm_headis locked to 6 bits to prevent reasoning boundary collapse. - Conversion (
convert_model.py): Compiled with-rcp recipe.yaml -cd cal_trace.safetensors -ss 4096.
Download & Usage
# Download via huggingface-cli
hf download Luni/Nanbeige-4.3-33B-Heretic-EXL3-SC --revision 3.50bpw --local-dir ./Nanbeige-4.3-33B-Heretic-EXL3-SC-3.50bpw
Model Lineage & Related Repositories
- Base Model: Nanbeige/Nanbeige4.3-33B by Nanbeige Team (Apache-2.0)
- Heretic BF16: Luni/Nanbeige-4.3-33B-Heretic
- Stock EXL3: Luni/Nanbeige-4.3-33B-EXL3
- Heretic EXL3 Master Suite: Luni/Nanbeige-4.3-33B-Heretic-EXL3
- Speculative Drafter: Luni/Nanbeige-4.3-33B-DFlash2
- Heretic GGUF & GGUF-IQ Suite: Luni/Nanbeige-4.3-33B-Heretic-GGUF
- Heretic GSQ-RCO Non-Uniform GGUF Suite: Luni/Nanbeige-4.3-33B-Heretic-GSQ-RCO-GGUF