LoRA adapter for Qwen2.5-VL-7B-Instruct, trained with supervised fine-tuning (SFT), seed 1, with checkpoints from step 50 to step 300 (training dynamics).
Released with the paper Accuracy Is Not Evidence: Measuring and Training Evidence Dependence in Video Temporal Grounding (ICLR 2027 submission, under review). Code, data identifiers and evaluation scripts are in iclropen/evidence-audit-code.
It appears in Figure 9 of the paper.
The adapter behind the paper's numbers was not kept. This one was trained again with the same code, data, settings, seed and example order, and the last section compares the two. All numbers in the paper come from the original run.
The root folder is step 300, and checkpoints/step-NNN/ hold the run after NNN of its 300 steps (8 pairs per step). Figure 9 plots the original run at the same steps.
| Folder | Contents | transformers |
|---|---|---|
| root | final adapter, step 300 | 4.51 (tested 4.51.3, peft 0.20) |
checkpoints/step-050/ |
checkpoint at step 50 | 4.51 (tested 4.51.3, peft 0.20) |
checkpoints/step-100/ |
checkpoint at step 100 | 4.51 (tested 4.51.3, peft 0.20) |
checkpoints/step-150/ |
checkpoint at step 150 | 4.51 (tested 4.51.3, peft 0.20) |
checkpoints/step-200/ |
checkpoint at step 200 | 4.51 (tested 4.51.3, peft 0.20) |
checkpoints/step-250/ |
checkpoint at step 250 | 4.51 (tested 4.51.3, peft 0.20) |
Adapters saved under transformers 4.51 and 5 use different module names. Loading one under the other version raises no error but leaves the LoRA weights unused, so use the version given here. tools/check_lora_keys.py in the code repository checks that an adapter actually loaded.
The transformers 4.51 runs used gradient checkpointing, under which the LoRA weights of the vision tower received no gradient. They are zero, so these adapters change the language model only.
# pip install "transformers==4.51.3" "peft==0.20.0" qwen-vl-utils
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base = "Qwen/Qwen2.5-VL-7B-Instruct"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(base)
model = PeftModel.from_pretrained(model, "iclropen/evidence-audit-qwen7b-sft-s1-dynamics")
The evaluation and audit scripts of the code repository take --model <base> --adapter <this repository or folder>.
Supervised fine-tuning with the same trainer and data as the placebo-controlled models: the answer likelihood only, with the margin terms switched off.
| Setting | Value |
|---|---|
| Training pairs | 2,400 from the Charades-STA training split |
| Optimizer steps | 300 |
| Gradient accumulation | 8 |
| Learning rate | 5e-6 |
| Seed | 1 |
| LoRA | r=16, alpha=32, dropout 0.05, on q/k/v/o and gate/up/down projections |
| Video input | 2 fps, at most 768 frames |
| Prompt template | timer1_default |
| Trainer | code/27_papo_train_qcdro_v3seln.py |
Both adapters were audited against Qwen2.5-VL-7B-Instruct on the first 200 Charades-STA test questions, with the same code under transformers 4.51.3. Γ, the paper's measure of evidence dependence, is in nats per token over the questions both models localise (n), with 95% video-clustered bootstrap intervals. R@0.5 is measured on all 3,720 test questions at 2 fps with greedy decoding, paired by question.
| Retrained (this repository) | Original run | |
|---|---|---|
| Γ, mask | -0.098 [-0.160, -0.037], n=109 | -0.094, n=108 |
| Γ, swap | -0.058 [-0.100, -0.016], n=109 | -0.036, n=108 |
| R@0.5 change against the base | +8.5 points [7.2, 9.8] (59.2% vs 50.7%) | – |
| Answers with a parsable interval | 100.0% | – |
The original-run values are from the step-300 readout of the original run behind Figure 9.
Anonymous. Accuracy Is Not Evidence: Measuring and Training Evidence Dependence in Video Temporal Grounding. ICLR 2027 submission, under review.
adapter_model.safetensors 190 MBcheckpoints/step-050/adapter_model.safetensors 190 MBcheckpoints/step-100/adapter_model.safetensors 190 MBcheckpoints/step-150/adapter_model.safetensors 190 MBcheckpoints/step-200/adapter_model.safetensors 190 MBcheckpoints/step-250/adapter_model.safetensors 190 MBQwen/Qwen2.5-VL-7B-Instruct