CodeThink-V4-Qwen3-8B GGUF

This repository provides GGUF quantizations for modrill/CodeThink-V4-Qwen3-8B.

CodeThink-V4-Qwen3-8B is an 8.19B-parameter code reasoning research model built on Qwen/Qwen3-8B-Base. It was trained using V4 LoRA SFT distillation on paired thinking traces from Qwen/Qwen3-30B-A3B-Thinking-2507, then merged to full weights. The model is tuned for programmatic problem solving, algorithmic code generation, and step-by-step reasoning inside <think> blocks.

All GGUF files were converted from the original merged safetensors weights using llama.cpp at native BF16 precision, then quantized into standard k-quant variants.

---

Available Files and Quantizations

---

Technical Specifications

• Base Architecture: Qwen3ForCausalLM (model_type: qwen3)
• Parameters: ~8.19B
• Layers: 36
• Hidden Size: 4096
• Intermediate Size: 12288
• Attention Heads: 32 query heads, 8 key/value heads (Grouped Query Attention)
• Head Dimension: 128
• Context Window: 32,768 tokens
• Vocabulary Size: 151,936 tokens
• Native Precision: bfloat16
• License: Apache 2.0

---

Benchmark Results (DEV256)

Evaluated on the 256-problem LiveCodeBench-derived development split (seed 3407, thinking mode, without <think> prefill, max generation ~32k tokens, pass@1):

---

Prompt Format

CodeThink-V4-Qwen3-8B uses ChatML markup. To trigger reasoning traces, prompt the assistant turn and allow the model to emit its own <think> block (do not prefill <think> manually):

<|im_start|>system
You are an expert Python programmer. You will be given a question (problem specification) and will generate a correct Python program that matches the specification and passes all tests. You will NOT return anything except for the program.<|im_end|>
<|im_start|>user
Write a Python function to solve the two-sum problem efficiently.<|im_end|>
<|im_start|>assistant

---

How to Use

1. With llama.cpp

Run interactive generation with llama-cli:

llama-cli -m ./CodeThink-V4-Qwen3-8B-Q4_K_M.gguf \
    -p "<|im_start|>system\nYou are an expert Python programmer.<|im_end|>\n<|im_start|>user\nWrite a Python function to check if a string is a palindrome.<|im_end|>\n<|im_start|>assistant\n" \
    -n 1024 \
    -c 4096 \
    --temp 0.6 \
    --top-p 0.95 \
    --top-k 20

Launch an OpenAI-compatible local API server:

llama-server -m ./CodeThink-V4-Qwen3-8B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 8192

2. With Ollama

Create a file named Modelfile in the same directory:

FROM ./CodeThink-V4-Qwen3-8B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER top_k 20

Build and run the model in Ollama:

ollama create codethink-8b -f Modelfile
ollama run codethink-8b

3. With LM Studio

1. Copy the desired .gguf file into your local LM Studio models directory.
2. Select CodeThink-V4-Qwen3-8B in the model selector.
3. Choose ChatML as the preset prompt template.

---

Attribution & Credits

• Base weights and architecture by Qwen Team.
• Fine-tuning and distillation by modrill.
• GGUF conversions created by Pink Pixel.