evidence-multi-rlm-4b-grpo-step100
GRPO-trained multi-paper evidence-extraction RLM. 100 GRPO steps on top of
alphaXiv/evidence-multi-rlm-sft-4b,
trained with SkyRL using an LLM-judge reward.
Gathered from the FSDP world_size=8 training checkpoint
(final-judge/global_step_100) into a single HF directory. Loads exactly like
the SFT checkpoints.
Eval
Multi-paper evidence validation set (32 prompts), greedy decoding, judged by
openai/gpt-5.4-mini. Reward is (judge_precision + judge_recall) / 2.
| Model | avg_score | pass_at_1 |
|---|---|---|
| this checkpoint | 0.52 +/- 0.01 (3 runs) | 0.94-0.97 |
evidence-multi-rlm-sft-4b (no RL) |
0.4172 | 0.9688 |
Not bitwise reproducible even at temperature 0: vLLM continuous batching is batch-composition dependent and the judge is not fully deterministic.
Behaviour: submits a well-formed answer on 31/32 prompts, all via FINAL_VAR.
Uses the intended parent/child decomposition (select papers -> dispatch child
agents per paper -> aggregate spans). Known failure modes: occasional
repetition to the length cap (3/531 turns), and sometimes prompting children
with a question rather than an extraction instruction, which yields answers
instead of verbatim spans.
Usage
from transformers import AutoModelForImageTextToText, AutoTokenizer
model = AutoModelForImageTextToText.from_pretrained("alphaXiv/evidence-multi-rlm-4b-grpo-step100", dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("alphaXiv/evidence-multi-rlm-4b-grpo-step100")
The vision tower is carried over from the base model and is unused by the text-only evidence task.