gpt-oss-20b PS-IQ4_NL (Peasant Smith)
A quality-focused GGUF requant of openai/gpt-oss-20b, built entirely from the original OpenAI weights — no third-party quant as source, no MXFP4 shortcut beyond what OpenAI itself ships.
Why IQ4_NL and not IQ2_XXS?
This model is geometrically incapable of IQ2_XXS/Q2_K/Q6_K: every quantizable tensor is 2880 columns wide (n_embd 2880 % 256 = 64), and all 256-element-block quant formats hard-require cols % 256 == 0. The 32-element-block formats (IQ4_NL, Q5_0, Q8_0) are the highest-fidelity floor this architecture permits. This is the same finding documented for gpt-oss-120b; published "Q2_K" files for gpt-oss models by third parties are silently-fallen-back 4-bit-size files.
Provenance chain (100% original weights)
openai/gpt-oss-20bofficial safetensors (experts natively MXFP4 — the only form in which they exist anywhere)- → GGUF master via
convert_hf_to_gguf.py(experts preserved in original MXFP4, everything else BF16→F16) - → experts decoded MXFP4→F16 with a custom pass validated bit-exact against the OCP MXFP4 spec decode (
maxdiff 0.000e+00on independent reference decode; e8m0 scale range 117–129, max |value| ≈ 24) - →
llama-quantizetensor-split requant (below)
No fidelity was lost decoding: the decode reproduces the original MXFP4 weight values exactly; the only quantization step is the final split.
Tensor split (verified from output file)
| Class | Tensors | Type |
|---|---|---|
| Expert gate/up weights | 48 | IQ4_NL |
| Expert down weights | 24 | Q5_0 |
| Attention, router, output, shared projections | 98 | Q8_0 |
| Norms and biases | 289 | F32 (kept) |
Down-projection errors land directly on the residual stream, so down gets one full step above gate/up. Attention and output stay at Q8_0. Imatrix is not consumed by 32-block types, so none was used.
File: gpt-oss-20b-PS-IQ4_NL.gguf — 13.50 GB (≈5.2 bits/param vs upstream 20.9B total params)
SHA-256: 22f66ec4fd0b6a7e69c490456234e132775b2c210926dc124cd46b7d8b560c89
Measured quality checks
| Test | Result |
|---|---|
| Greedy smoke ("The capital of France is") | ✅ correct, with clean reasoning block |
| LRU cache code task (8k ctx, greedy) | ✅ correct OrderedDict implementation, O(1) get/put, eviction |
| Needle recall ("9137 steps", 8k ctx) | ✅ exact answer |
| Perplexity | not measured — gpt-oss is harmony-format-only; raw-text PPL is invalid on this family (the F16 itself produces non-finite loss on wikitext) |
Measured speed (2× RTX 3060 12GB, llama.cpp master, -ngl 99 -ts 1,1, ctx 8192, greedy)
| Metric | Value |
|---|---|
| Prompt processing | 229–389 t/s |
| Generation | 86–91 t/s |
Fully VRAM-resident on 2× 12 GB. For 12 GB single-GPU machines, offload experts: llama-server -m ... -ngl 99 --n-cpu-moe 16 (documented fallback; not the tested config).
Usage
Harmony chat format is required (use the jinja chat template shipped in this repo's tokenizer files / llama.cpp built-in). No MTP head exists for this architecture.
Limitations
- Text-only (this GGUF carries no vision tower — the base model has none)
- Same knowledge/cutoff and safety profile as the upstream openai/gpt-oss-20b
- 4-bit experts: a small but real perplexity/quality delta vs the MXFP4 original exists by construction; task-level checks above all passed
Credit
Upstream model by OpenAI (Apache-2.0). Requant by Peasant Smith.