Mellum2 Thinking (SFT)

[!Note] Research artifact: the SFT-only intermediate checkpoint of the Thinking training pipeline. Use it to study or build on the reasoning SFT stage in isolation, run post-training experiments (preference tuning, RLVR, etc.), or compare against the final RL-tuned variant. For production-quality reasoning use Thinking instead.

Mellum2 Thinking (SFT) Highlights

Mellum2 Thinking (SFT) is the supervised-fine-tuning-only intermediate checkpoint of the Mellum2 Thinking training pipeline, released by JetBrains as a research artifact.

The model uses a Mixture-of-Experts architecture with 64 experts and activates 8 experts per token. It uses a combination of sliding-window and full attention layers, with a context length of 131,072 tokens.

This checkpoint was produced from Mellum2-12B-A2.5B-Base with three epochs of SFT on a corpus of reasoning-trace-bearing data. The model emits its reasoning inside <think>...</think> blocks. It is the starting point of the RLVR stage that produces the final Mellum2-12B-A2.5B-Thinking.

Mellum2 Model Family

This repository contains one checkpoint from the Mellum2 family.

Model Overview

Mellum2 Thinking (SFT) has the following features:

• Number of Layers: 28
• Hidden Size: 2304
• Intermediate Size: 7168
• MoE Intermediate Size: 896
• Number of Experts: 64
• Number of Activated Experts: 8
• Number of Attention Heads (GQA): 32 for Q and 4 for KV
• Context Length: 131,072
• Sliding Window: 1,024
• Vocabulary Size: 98,304
• Precision: bfloat16

Serving with vLLM

# Without tool calling
vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking-SFT \
  --max-model-len 131072 \
  --reasoning-parser qwen3

# With tool calling
vllm serve JetBrains/Mellum2-12B-A2.5B-Thinking-SFT \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Quickstart

Text-Only Input

from openai import OpenAI
# Configured by environment variables
client = OpenAI()

messages = [
    {"role": "user", "content": "Is 1024 a power of 2? Explain your reasoning."},
]

chat_response = client.chat.completions.create(
    model="JetBrains/Mellum2-12B-A2.5B-Thinking-SFT",
    messages=messages,
    max_tokens=81920,
    temperature=0.6,
    top_p=0.95,
    extra_body={
        "top_k": 20,
    },
)
print("Chat response:", chat_response)

Evaluation

Evaluation results are available in the model card. All values are self-reported by JetBrains.

For more details, see the Mellum2 Technical Report.

License

Released under the Apache 2.0 license.