Nanbeige-4.3-33B-Heretic-EXL3

ExLlamaV3 (EXL3) quantizations of Luni/Nanbeige-4.3-33B-Heretic (an abliterated and de-slopped build of Nanbeige/Nanbeige4.3-33B).

Refusal and preachy alignment vectors removed while retaining full native <think> chain-of-thought reasoning.
Reduced repetitive RLHF prose patterns and moralizing lectures.
Patched "add_bos_token": false in tokenizer_config.json to eliminate upstream duplicate <|im_start|> BOS token emission when applying chat templates.
Features both standard LDLQ quants and turboderp Self-Calibrated (SC) quants.

Released Branches

Standard Tiers

Branch	Avg BPW	Head (`-hb`)	Embed (`-eb`)	Size	Target VRAM	Notes
`5.00bpw`	`5.00`	`6`	`16` (FP16)	`21.0 GB`	`32 GB`	High precision (`64k+` context)
`4.50bpw`	`4.50`	`6`	`16` (FP16)	`19.3 GB`	`24 GB`	KLD-optimal Pareto merge (`measure_model` + `optimize_model`)
`4.00bpw`	`4.00`	`6`	`16` (FP16)	`17.6 GB`	`24 GB / 32 GB`	Standard 4-bit (`32k-64k+` context)

Calibrated with self-sampled in-domain <think> traces and per-tensor LDLQ noise attribution:

Branch	Target VRAM	Size	Notes
`SC_3.50bpw`	16 GB VRAM	`14.0 GB`	Near-lossless reasoning preservation (KLD ~0.035)
`SC_3.00bpw`	16 GB VRAM	`12.0 GB`	Room for 32k context on 16 GB cards
`SC_2.50bpw`	12 GB - 16 GB VRAM	`10.0 GB`	Native 2.5 bpw codebook sweet spot
`SC_2.25bpw`	12 GB VRAM	`9.4 GB`	Extreme budget fit for RTX 3060 12GB

# Download via huggingface-cli
hf download Luni/Nanbeige-4.3-33B-Heretic-EXL3 --revision 4.00bpw --local-dir ./Nanbeige-4.3-33B-Heretic-EXL3-4.00bpw