🚚 Supply Chain Exception Resolution — GRPO Fine-tuned

Qwen/Qwen2.5-3B-Instruct fine-tuned with Group Relative Policy Optimisation (GRPO) on the supply_chain_exception_env OpenEnv benchmark (Meta PyTorch Hackathon, April 2026).

Given a live supply chain state — pending orders, warehouse inventory, available routes, supplier options, and budget — the model emits a single structured JSON action that maximises customer fulfilment outcomes across 8 tasks spanning easy, medium, and hard difficulty.

---

📊 Training Results

---

📈 Training Curves

Learning Curve with Confidence Band — GRPO reward rises steadily from 0.2 → 1.85 over 125 steps. The blue dashed line marks the heuristic baseline target (0.935). The model surpasses the baseline by step 10 and maintains that lead throughout training.

Policy Entropy over Training — Entropy stays non-zero and oscillates between 0.1–0.75 throughout all 125 steps, confirming the policy never collapsed. The right scatter shows high-reward episodes are achievable at all entropy levels — the agent learned when to be decisive and when to explore.

Episode Score per Task vs Heuristic Baseline — the trained model matches or exceeds the heuristic on 7 of 8 tasks, with the largest gains on medium and hard difficulty tasks.

Score by Difficulty Tier (mean ± std) — the model generalises across easy, medium, and hard tasks, with consistent scores and controlled variance.

---

🚀 Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import json, torch

model_id  = "rajveer43/supply-chain-grpo-qwen2.5-3b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model     = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

SYSTEM_PROMPT = """You are a supply chain exception resolver.
Analyze the current environment state and emit EXACTLY ONE JSON action.

Valid action types:
- reroute_shipment      (shipment_id, new_route_id)
- reallocate_inventory  (from_warehouse, to_warehouse, item_sku, quantity)
- expedite_order        (order_id, supplier_id, premium_cost)
- substitute_supplier   (order_id, new_supplier_id)
- backorder_customer    (customer_id, order_id, estimated_days)
- prioritize_order      (order_id, priority_level)
- cancel_or_hold        (order_id, action, reason)

Respond with valid JSON only. No explanation."""

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": """
Pending orders:
- ORD-P1:  200 units  priority=critical  customer=CUST-VIP   SLA=1 day
- ORD-MW1: 150 units  priority=high      customer=CUST-002   SLA=3 days

WH-North has 300 excess units of ITEM-ALPHA.
WH-South is undersupplied (needs 200 units).
Budget remaining: $9,200
What action should we take?"""},
]

inputs  = tokenizer.apply_chat_template(
    messages, return_tensors="pt", add_generation_prompt=True
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=128, do_sample=False)
action  = tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True)
print(json.loads(action))
# → {"action_type": "reallocate_inventory",
#    "payload": {"from_warehouse": "WH-North", "to_warehouse": "WH-South",
#                "item_sku": "ITEM-ALPHA", "quantity": 200}}