gpt-oss-120b-VISION-v2 · GGUF

OpenAI's larger open-weight reasoning model with its released weights unchanged, a corrected chat template that fixes malformed tool-call output, and image input.

OpenAI's original gpt-oss-120b weights, unchanged, packed as GGUF for llama.cpp and SoAI.

---

What's new in v2

Its weights are identical to the first release, byte for byte. This release replaces the embedded chat template, fixing malformed output and invalid JSON that gpt-oss could produce around tool calls.

With OpenAI's original template, gpt-oss occasionally wrote a damaged channel header around a tool call, such as <|channel|>comment.... llama.cpp could not parse such replies, so clients received raw text or an error instead of a tool call with valid JSON arguments. gpt-oss users report the same malformed JSON with Ollama. The corrected template opens each reply inside gpt-oss's analysis channel, where it always begins, so the model never has to write that header itself. In our testing, llama.cpp parsed every reply and every tool call arrived with valid JSON arguments.

It also corrects how earlier tool calls and results are written back into a conversation:

If you already have the first release's files (gpt-oss-120b-VISION-GGUF), keep them and load the corrected template instead of downloading again: get chat_template.jinja from this repository and start llama-server with -m gpt-oss-120b-MXFP4-00001-of-00002.gguf --jinja --chat-template-file chat_template.jinja.

At a glance

Choosing a file

[!TIP] This repository has one model build because OpenAI post-trained this model with its expert weights already in MXFP4 and ran its published evaluations in that format. This file keeps that format, so it is the model as OpenAI released and evaluated it.

Running it takes memory for the file and for its context, which grows with the context length you set. llama.cpp can run this model on GPU, on CPU, or split its layers between both.

Images need the mmproj-gpt-oss-120b-VISION-v2-f16.gguf add-on as well, loaded with --mmproj. llama-server -hf downloads it automatically.

Quick start

With llama.cpp, this command starts a local server with a built-in chat UI and an OpenAI-compatible API:

llama-server -hf SoAIHQ/gpt-oss-120b-VISION-v2-GGUF:MXFP4

Open http://localhost:8080 to chat, or send requests to the API:

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Explain GGUF in one sentence."}],
  "chat_template_kwargs": {"reasoning_effort": "high"}
}'

Download the files to a folder first

hf download SoAIHQ/gpt-oss-120b-VISION-v2-GGUF --include "gpt-oss-120b-VISION-v2-MXFP4-*.gguf" "mmproj-*" --local-dir gpt-oss-120b-VISION-v2-GGUF
llama-server -m gpt-oss-120b-VISION-v2-GGUF/gpt-oss-120b-VISION-v2-MXFP4-00001-of-00002.gguf \
  --mmproj gpt-oss-120b-VISION-v2-GGUF/mmproj-gpt-oss-120b-VISION-v2-f16.gguf

gpt-oss only works with its harmony prompt format. Its corrected chat template, described above, is embedded in the GGUF, so runtimes apply it automatically and llama.cpp parses tool calls back into OpenAI-compatible tool_calls.

Recommended settings

OpenAI recommends these values in the original model card. Reasoning is always on, and low effort gives quick answers. Allow at least 16K output tokens to leave room for long answers after reasoning.

In llama-server, you set reasoning per request with "chat_template_kwargs": {"reasoning_effort": "high"}.

Agents and tool calling

gpt-oss plans in its reasoning. During a sequence of tool calls, send each step's reasoning back with its assistant message, as reasoning_content in llama-server's OpenAI-compatible API. Once the model gives its final answer, the template drops that turn's reasoning from later prompts on its own, as OpenAI's harmony guide describes.

{"role": "assistant", "content": "", "reasoning_content": "Need the file list first.",
 "tool_calls": [{"id": "call_1", "type": "function", "function": {"name": "list_directory", "arguments": "{\"path\": \".\"}"}}]}

Sending it back keeps the model's plan intact from one step to the next, and SoAI does this during tool sequences.

Tested on SoAI

We tested this chat template on SoAI with gpt-oss as a full agent, in Execute mode with all of SoAI's default tools and recommended settings. In every run, llama.cpp parsed every reply, and the model based its final report on tools it had actually called.

How these files are made

Its mixture-of-experts weights are copied in MXFP4, the format OpenAI post-trained and evaluated them in, and every other tensor keeps its released BF16 precision. Nothing is requantized: conversion changes only the container, and weights stay as released.

Before packaging, the build checks that chat, thinking and tool-call markers are stored as special tokens. A conversion that imports those markers as plain text silently breaks turns, thinking and tool calls, so in that case the build stops instead of publishing anything. After those checks, the build swaps in the corrected chat template described above and changes nothing else. This template is also published here as chat_template.jinja.

The original model reads text only. This repository adds image input through a vision encoder and projector from OpenGVLab/InternVL3_5-GPT-OSS-20B-A4B-Preview (revision aaabe6aa48), which its authors trained for this model family. The mmproj-*.gguf file holds only those vision tensors, and its language model is the original release, so text, reasoning and tool calling work identically with or without it. Its vision weights keep the donor's license, described on its model card.

For best image answers, put your image before your question in one message.

All files and SHA-256 checksums

The full checksums are in SHA256SUMS.

Provenance

License and credits

These files are distributed under the original model's license (apache-2.0), and all of its terms continue to apply. The license and original model card also cover this model's capabilities, training data, benchmarks and limitations.

OpenAI designed and trained this model, and SoAI only repackages its published weights. Conversion uses llama.cpp by Georgi Gerganov and ggml contributors.

Run it with SoAI

Your models, your hardware, one software suite. Download SoAI for Windows, macOS, Linux or Android.

_{Packaged by SoAI.}