OxCoder-9B GGUF

This repository provides GGUF quantizations for OxCoder-9B, a 9-billion parameter coding model optimized for long-horizon agentic software engineering and terminal tasks.

OxCoder-9B is built on the Qwen 3.5 9B foundation and trained on agentic trajectories from frontier coding agents (including Fable-5.1 and GLM-5.3 traces across Claude Code, OpenCode, and Codex). It incorporates read-before-write inspection, tool call error recovery, and minimal edit diff generation.

All GGUF files were converted from the original safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

Available Files and Quantizations

File	Quant Type	Size	Description / Recommendation
`OxCoder-9B-BF16.gguf`	BF16	16.69 GB	Full precision base conversion. Highest fidelity reference weights.
`OxCoder-9B-Q8_0.gguf`	Q8_0	8.87 GB	Near-lossless 8-bit quantization. Recommended for production coding when RAM permits.
`OxCoder-9B-Q6_K.gguf`	Q6_K	6.85 GB	High quality retention with minimal degradation. Excellent accuracy-to-size balance.
`OxCoder-9B-Q5_K_M.gguf`	Q5_K_M	6.02 GB	Balanced quantization. Strong coding reasoning with moderate memory usage.
`OxCoder-9B-Q4_K_M.gguf`	Q4_K_M	5.24 GB	Recommended default. Fast, responsive, and fits easily into typical consumer GPUs.
`OxCoder-9B-Q3_K_M.gguf`	Q3_K_M	4.31 GB	Compact footprint when memory headroom is strictly limited.
`OxCoder-9B-Q2_K.gguf`	Q2_K	3.56 GB	Maximum compression. Noticeable quality loss, intended for memory-constrained testing.

Architecture Requirements

OxCoder-9B uses the Qwen 3.5 architecture (Qwen3_5ForConditionalGeneration / qwen35 in llama.cpp), which incorporates hybrid linear attention layers interspersed with full self-attention and interleaved MRoPE rotary position embeddings.

Ensure your llama.cpp build or downstream runtime (such as Ollama, LM Studio, or Jan) includes Qwen 3.5 support.

Quickstart

1. Terminal Inference with llama.cpp

Run the model interactively using llama-cli:

llama-cli -m ./OxCoder-9B-Q4_K_M.gguf \
    -p "<|im_start|>user\nWrite a Python script to parse git commit history and summarize author stats.<|im_end|>\n<|im_start|>assistant\n" \
    -n 1024 \
    -c 8192 \
    --temp 0.6

To run a local OpenAI-compatible API server:

llama-server -m ./OxCoder-9B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 16384

2. Ollama

Create a file named Modelfile in the same directory:

FROM ./OxCoder-9B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6

Then register and start the model:

ollama create oxcoder-9b -f Modelfile
ollama run oxcoder-9b

3. LM Studio

Place any .gguf file from this folder into your LM Studio models directory.
Search for OxCoder-9B in your local models tab.
Select ChatML as the prompt format preset.

Benchmark Highlights

Benchmark	OxCoder-9B	Qwen3.5-9B	SWE-bench Verified / Task
Terminal-Bench 2.1 (Claude Code)	50.8	18.9	Agentic terminal coding
Terminal-Bench 2.1 (Terminus-2)	49.6	21.3	Agentic terminal coding
SWE-bench Verified	73.5	53.2	Agentic issue resolution
SWE-bench Pro	49.1	31.3	Real-world software engineering
NL2Repo	36.2	16.2	Repo-level code generation
GPQA Diamond	86.9	82.5	Scientific and logic reasoning

Model Details

Base model: Qwen/Qwen3.5-9B
Architecture: Qwen 3.5 (Qwen3_5ForConditionalGeneration)
Parameters: ~9 Billion
Native context length: 262,144 tokens
Vocabulary size: 248,320 tokens
License: Apache 2.0

Attribution & Credits

Base model architecture by Qwen. Training trajectories distilled from frontier coding agent traces.

GGUF conversions produced by Pink Pixel.

Made with 💖 by Pink Pixel

sizzlebop/OxCoder-9B-GGUF