🧠 Qwen2.5 + GRPO — Structured Reasoning Model

A fine-tuned version of Qwen2.5 trained with GRPO (Group Relative Policy Optimization) to reason before it answers — not just pattern-match.

---

Overview

Most LLMs simulate reasoning by mimicking patterns seen during training. This model is different: it builds a real cognitive path on every response by following a strict, verifiable reasoning protocol enforced through reinforcement learning.

Every response goes through three mandatory stages:

This isn't chain-of-thought bolted on top — the reasoning protocol is baked in via RL.

System Prompt

SYSTEM_PROMPT = """
You are an AI assistant that MUST produce structured reasoning.
Your response MUST EXACTLY follow this format:
<think>
<planning>
...
</planning>
<monitoring>
...
</monitoring>
<evaluation>
...
</evaluation>
</think>
<output>
...
</output>
FORMAT RULES:
1. The <think> block must contain exactly three sections in this order:
   <planning>, <monitoring>, <evaluation>
2. Each section must contain detailed reasoning in full sentences.
3. Minimum reasoning length:
   - <planning>: at least 40 tokens
   - <monitoring>: at least 80 tokens
   - <evaluation>: at least 40 tokens
4. The <monitoring> section MUST show explicit reasoning steps,
   including calculations, derivations, or logical deductions.
5. Generic placeholder phrases are forbidden, including:
   - "analyze the problem"
   - "determine the strategy"
   - "verify the solution"
   - "check correctness"
6. The reasoning must explicitly reference values, equations,
   or logical relationships from the problem.
7. The <output> section must contain ONLY the final answer.
INVALID RESPONSES:
Responses will be rejected if they contain:
- Empty sections
- Bullet point placeholders
- Generic reasoning
- Missing calculations when required
- Incorrect tag order
The format must always be strictly respected.
"""

---

Usage

from vllm import SamplingParams

def generate_response(question, choices):
    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT
        },
        {
            "role": "user",
            "content": (
                f"Examine the following question and select the right answer from given options.\n"
                f"The output must be only the number of the option.\n"
                f"Question: {question}\n"
                f"Provided options: {choices}\n"
            )
        }
    ]

    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
        return_tensors="pt",
    )

    sampling_params = SamplingParams(
        temperature=0.8,
        top_p=0.95,
        max_tokens=1024,
    )

    output = model.fast_generate(
        [inputs],
        sampling_params=sampling_params,
        lora_request=None,
    )[0].outputs[0].text

    return output

---

What Makes This Different

---

MMLU Benchmark Results

We selected random 100 samples from each subsets of MMLU dataset. Performance across a range of MMLU subject categories:

🎓 College Courses

🧑‍💼 Professional

🏫 High School Courses

Results reflect accuracy on MMLU multiple-choice questions using the structured reasoning protocol described above.

---

Training

• Base model: Qwen2.5
• Training method: GRPO (Group Relative Policy Optimization)
• Objective: Enforce structured reasoning as a non-negotiable output constraint, not a post-hoc addition