中文Sign up

peasantsmith/gpt-oss-20b-PS-IQ4_NL-GGUF

Verified creator peasantsmith verified
🤗 Hugging Face sourceapache-2.014 GBGGUF✓ 1 checksumupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo peasantsmith/gpt-oss-20b-PS-IQ4_NL-GGUF ./model-folder
Needs a seeder →

gpt-oss-20b PS-IQ4_NL (Peasant Smith)

A quality-focused GGUF requant of openai/gpt-oss-20b, built entirely from the original OpenAI weights — no third-party quant as source, no MXFP4 shortcut beyond what OpenAI itself ships.

Why IQ4_NL and not IQ2_XXS?

This model is geometrically incapable of IQ2_XXS/Q2_K/Q6_K: every quantizable tensor is 2880 columns wide (n_embd 2880 % 256 = 64), and all 256-element-block quant formats hard-require cols % 256 == 0. The 32-element-block formats (IQ4_NL, Q5_0, Q8_0) are the highest-fidelity floor this architecture permits. This is the same finding documented for gpt-oss-120b; published "Q2_K" files for gpt-oss models by third parties are silently-fallen-back 4-bit-size files.

Provenance chain (100% original weights)

  1. openai/gpt-oss-20b official safetensors (experts natively MXFP4 — the only form in which they exist anywhere)
  2. → GGUF master via convert_hf_to_gguf.py (experts preserved in original MXFP4, everything else BF16→F16)
  3. → experts decoded MXFP4→F16 with a custom pass validated bit-exact against the OCP MXFP4 spec decode (maxdiff 0.000e+00 on independent reference decode; e8m0 scale range 117–129, max |value| ≈ 24)
  4. → llama-quantize tensor-split requant (below)

No fidelity was lost decoding: the decode reproduces the original MXFP4 weight values exactly; the only quantization step is the final split.

Tensor split (verified from output file)

Class Tensors Type
Expert gate/up weights 48 IQ4_NL
Expert down weights 24 Q5_0
Attention, router, output, shared projections 98 Q8_0
Norms and biases 289 F32 (kept)

Down-projection errors land directly on the residual stream, so down gets one full step above gate/up. Attention and output stay at Q8_0. Imatrix is not consumed by 32-block types, so none was used.

File: gpt-oss-20b-PS-IQ4_NL.gguf — 13.50 GB (≈5.2 bits/param vs upstream 20.9B total params) SHA-256: 22f66ec4fd0b6a7e69c490456234e132775b2c210926dc124cd46b7d8b560c89

Measured quality checks

Test Result
Greedy smoke ("The capital of France is") ✅ correct, with clean reasoning block
LRU cache code task (8k ctx, greedy) ✅ correct OrderedDict implementation, O(1) get/put, eviction
Needle recall ("9137 steps", 8k ctx) ✅ exact answer
Perplexity not measured — gpt-oss is harmony-format-only; raw-text PPL is invalid on this family (the F16 itself produces non-finite loss on wikitext)

Measured speed (2× RTX 3060 12GB, llama.cpp master, -ngl 99 -ts 1,1, ctx 8192, greedy)

Metric Value
Prompt processing 229–389 t/s
Generation 86–91 t/s

Fully VRAM-resident on 2× 12 GB. For 12 GB single-GPU machines, offload experts: llama-server -m ... -ngl 99 --n-cpu-moe 16 (documented fallback; not the tested config).

Usage

Harmony chat format is required (use the jinja chat template shipped in this repo's tokenizer files / llama.cpp built-in). No MTP head exists for this architecture.

Limitations

  • Text-only (this GGUF carries no vision tower — the base model has none)
  • Same knowledge/cutoff and safety profile as the upstream openai/gpt-oss-20b
  • 4-bit experts: a small but real perplexity/quality delta vs the MXFP4 original exists by construction; task-level checks above all passed

Credit

Upstream model by OpenAI (Apache-2.0). Requant by Peasant Smith.