ggml-org/gpt-oss-20b-GGUF

Source: 🤗At risk🏴‍☠️ Rescued by: Sherpatext-generationapache-2.020B activated15 GBGGUF✓ 3 checksumsupdated 2d ago

gpt-oss-20b

Run it in the Llama app: https://llama.app

Or from the terminal:

llama serve -hf ggml-org/gpt-oss-20b-GGUF

[!IMPORTANT] This model is automatically converted using https://github.com/ggml-org/convert

CommentsShare your thoughts and shape the future of sovereign AI.

@ggml-org is unclaimedSign in with the matching Hugging Face account or an organization member account to verify this handle.Claim @ggml-org

Files and sources

SourceAvailable on 🤗

Checksums3 recorded

TorrentMagnet listed

Swarm0 seeders reported

Weights

4-bit

8-bit

16-bit

Family

↑ Quantized from openai/gpt-oss-20b

Speed on real hardware

LocalMaxxing

Output speed for one request at a time, measured by people running this model on their own machines.

RTX 5080llama.cpp · MXFP4 · 1 run197 tok/s
```
# Remote endpoint: http://127.0.0.1:8081  servedModel: gpt-oss
```
View run on LocalMaxxing
2× RTX 3090✓llama.cpp · MXFP4 · 2 runs194 tok/s
```
# Remote endpoint: http://127.0.0.1:18082  servedModel: lmx-loop
```
View run on LocalMaxxing

RX 9070 XT✓llama.cpp · MXFP4 · 2 runs176 tok/s

llama-server -m gpt-oss-20b-MXFP4.gguf -fa on -c 8192 -np 1 --jinja (its --fit places what the card can't hold)

Ryzen AI Max 395✓custom · MXFP4 · 3 runs79.0 tok/s
```
loculus serve gpt-oss-20b-MXFP4.gguf --gpu
```
View run on LocalMaxxing

M1 Maxllama.cpp · MXFP4 · 1 run74.6 tok/s

/opt/homebrew/bin/llama-bench -m /Users/sgtpooki/Library/Caches/llama.cpp/gpt-oss-20b-MXFP4.gguf -p 512 -n 128

RTX 3060 Ti✓custom · MXFP4 · 3 runs27.3 tok/s
```
loculus serve gpt-oss-20b-MXFP4.gguf --gpu
```
View run on LocalMaxxing

Median of 13 community runs on 7 setups, from LocalMaxxing.