LoRA adapter for VideoChat-R1_7B, trained with the placebo-controlled objective at α = 0.25.
Released with the paper Accuracy Is Not Evidence: Measuring and Training Evidence Dependence in Video Temporal Grounding (ICLR 2027 submission, under review). Code, data identifiers and evaluation scripts are in iclropen/evidence-audit-code.
It appears in Tables 6, 7, 13, 14 and 16 and in Figures 1, 4 and 5 of the paper.
The adapter behind the paper's numbers was not kept. This one was trained again with the same code, data, settings, seed and example order, and the last section compares the two. All numbers in the paper come from the original run.
Load it with transformers 4.51 (tested 4.51.3, peft 0.20).
Adapters saved under transformers 4.51 and 5 use different module names. Loading one under the other version raises no error but leaves the LoRA weights unused, so use the version given here. tools/check_lora_keys.py in the code repository checks that an adapter actually loaded.
The transformers 4.51 runs used gradient checkpointing, under which the LoRA weights of the vision tower received no gradient. They are zero, so these adapters change the language model only.
# pip install "transformers==4.51.3" "peft==0.20.0" qwen-vl-utils
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
base = "OpenGVLab/VideoChat-R1_7B"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(base, torch_dtype=torch.bfloat16, device_map="auto")
processor = AutoProcessor.from_pretrained(base)
model = PeftModel.from_pretrained(model, "iclropen/evidence-audit-videochatr1-7b-pc-a025-s0")
The evaluation and audit scripts of the code repository take --model <base> --adapter <this repository or folder>.
The placebo-controlled objective of Section 4 of the paper. It keeps the answer likelihood, weighted by α, and adds two margins for each edit operator: the annotated answer must be more likely on the clean video, and on a video with a matched placebo edit outside the event, than on the video with the event edited. A hinge keeps the drop under the placebo edit within a bound set by the reference model.
| Setting | Value |
|---|---|
| Operator weights | Group-DRO |
| α | 0.25 |
| β | 0.1 |
| Placebo bound | placebo drop ≤ 1.25 × reference drop + 0.10 |
| Training pairs | 2,400 from the Charades-STA training split |
| Optimizer steps | 300 |
| Gradient accumulation | 8 |
| Learning rate | 5e-6 |
| Seed | 0 |
| LoRA | r=16, alpha=32, dropout 0.05, on q/k/v/o and gate/up/down projections |
| Edit operators | swap, mask, blur, tshift, shuffle (two per example) |
| Video input | 2 fps, at most 768 frames |
| Prompt template | vchat |
| Trainer | code/27_papo_train_qcdro_v3seln.py |
Both adapters were audited against VideoChat-R1_7B on the first 200 Charades-STA test questions, with the same code under transformers 4.51.3. Γ, the paper's measure of evidence dependence, is in nats per token over the questions both models localise (n), with 95% video-clustered bootstrap intervals.
| Retrained (this repository) | Original run | |
|---|---|---|
| Γ, mask | +1.61 [1.45, 1.78], n=135 | +1.63, n=136 |
| Γ, swap | +0.95 [0.79, 1.12], n=135 | – |
The original-run values are from Table 6 of the paper.
For comparison, evidence-audit-videochatr1-7b-pc-a05-s0 (α = 0.5) has Γ (mask) +1.43, n=134 in the same audit against the same base readout, and +1.43, n=132 in Table 6.
Anonymous. Accuracy Is Not Evidence: Measuring and Training Evidence Dependence in Video Temporal Grounding. ICLR 2027 submission, under review.
OpenGVLab/VideoChat-R1_7B