ggml-org/gpt-oss-120b-GGUF

Source: 🤗🏴‍☠️ Rescued by: Sherpatext-generationapache-2.0120B activated66 GBGGUF✓ 3 checksumsupdated 2d ago

gpt-oss-120b

Run it in the Llama app: https://llama.app

Or from the terminal:

llama serve -hf ggml-org/gpt-oss-120b-GGUF

[!IMPORTANT] This model is automatically converted using https://github.com/ggml-org/convert

Loading…

CommentsShare your thoughts and shape the future of sovereign AI.

@ggml-org is unclaimedSign in with the matching Hugging Face account or an organization member account to verify this handle.Claim @ggml-org

Files and sources

SourceAvailable on 🤗

Checksums3 recorded

TorrentMagnet listed

Swarm7 seeders reported

Weights

4-bit

8-bit

16-bit

Family

↑ Quantized from openai/gpt-oss-120b

Speed on real hardware

LocalMaxxing

Output speed for one request at a time, measured by people running this model on their own machines.

Ryzen AI Max 395✓custom · MXFP4 · 3 runs55.7 tok/s
```
loculus serve gpt-oss-120b-MXFP4.gguf --gpu
```
View run on LocalMaxxing

RX 9070 XT✓llama.cpp · MXFP4 · 2 runs13.2 tok/s

llama-server -m gpt-oss-120b-MXFP4.gguf -fa on -c 8192 -np 1 --jinja (its --fit places what the card can't hold)

Median of 5 community runs on 2 setups, from LocalMaxxing.