👑 🗓️ Qwen Scheduler 7B GRPO

A Language Model trained to schedule events with GRPO!

➡️ Read the full story in my blog post.

Find all the code in the GitHub repository.

The problem

Given a list of events and priorities, we ask the model to create a schedule that maximizes the total duration of selected events, weighted by priority.

In this setup, a priority event gets a weight of 2, and a normal event gets a weight of 1.

Example input

Events:

• Event A (01:27 - 01:42)
• Event B (01:15 - 02:30)
• Event C (15:43 - 17:43)

Priorities:

• Event B

Example output

<think>A detailed reasoning</think>
<schedule>
<event>
<name>Event B</name>
<start>01:15</start>
<end>02:30</end>
</event>
<event>
<name>Event C</name>
<start>15:43</start>
<end>17:43</end>
</event>
</schedule>

Evaluation

The training worked! The final 7B model improved on the task, significantly outperforming its base model and even a larger 14B model on the test set.

It got good at following the format and most rules.

However, it still struggles with preventing overlapping events, suggesting the reward function design for that specific constraint could be improved.

For more details, check out the blog post.

🔧 Training details

This model was trained from Qwen2.5-Coder-7B-Instruct, using Unsloth with QLoRA to save GPU.

I used GRPO Reinforcement Learning algorithm, meaning that the model only received prompts (no completions) and was guided during training by deterministic reward functions.

Training dataset.

Training (3 epochs) required about 23 hours on a single A6000 GPU.

Weight and Biases training report.

For complete details, check out the blog post.

🎮 Usage

This model was primarily an experiment in applying GRPO. I don't recommend using a Language Model for something that can be easily solved with deterministic programming.

If you want to try the model, you should use Unsloth. Unfortunately, all techniques to save the trained adapter to use it with other libraries (Transformers, vLLM) are currently not working. (See https://github.com/unslothai/unsloth/issues/2009). In short, the conversion appears to work, but you end up with a different model 🤷.

# ! pip install "unsloth==2025.3.19"

from unsloth import FastLanguageModel

# Load the model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="anakin87/qwen-scheduler-7b-grpo",
    max_seq_length=1500,
    load_in_4bit=False,  # False for LoRA 16bit
    fast_inference=False,
    gpu_memory_utilization=0.8,
)

FastLanguageModel.for_inference(model)  # Enable native 2x faster inference

SYSTEM_PROMPT = """You are a precise event scheduler.
1. First, reason through the problem inside <think> and </think> tags. Here you can create drafts, compare alternatives, and check for mistakes.
2. When confident, output the final schedule inside <schedule> and </schedule> tags. Your schedule must strictly follow the rules provided by the user."""

USER_PROMPT = """Task: create an optimized schedule based on the given events.

Rules:
- The schedule MUST be in strict chronological order. Do NOT place priority events earlier unless their actual start time is earlier.
- Event start and end times are ABSOLUTE. NEVER change, shorten, adjust, or split them.
- Priority events (weight = 2) carry more weight than normal events (weight = 1), but they MUST still respect chronological order.
- Maximize the sum of weighted event durations.
- No overlaps allowed. In conflicts, include the event with the higher weighted time.
- Some events may be excluded if needed to meet these rules.


You must use this format:  

<think>...</think>
<schedule>
<event>
<name>...</name>
<start>...</start>
<end>...</end>
</event>
...
</schedule>

---

Events:
- Digital detox meditation session (08:38 - 09:08)
- Small Language Models talk (08:54 - 10:09)
- Async Python talk (15:00 - 16:15)
- Live coding on Rust (17:23 - 19:23)

Priorities:
- Live coding on Rust
"""

messages = [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_PROMPT},
        ]


text = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=False,
)

# inference
res = model.generate(**tokenizer([text], return_tensors="pt").to("cuda"))
generated = tokenizer.decode(res[0], skip_special_tokens=True).rpartition("assistant\n")[-1]

print(generated)

# <think>A detailed reasoning</think>
# <schedule>
# <event>
# ...
# </event>
# ...
# </schedule>