Longformer QASPER Extractive Document QA

This model card is the upload template for the checkpoint produced by notebooks/complete_longformer_training_evaluation_pipeline.ipynb.

Base model

valhalla/longformer-base-4096-finetuned-squadv1

Project fine-tuning

The checkpoint is further fine-tuned by this project on a documented subset of QASPER v0.3 containing only answerable questions with one contiguous extractive span that can be located in the reconstructed paper text. Free-form, yes/no, unanswerable, unresolved, and multi-span annotations are excluded because this model predicts one contiguous span.

Evaluation

The model was evaluated on 200 contiguous-extractive examples from the QASPER validation set.

Training configuration

• Training examples: 803
• Validation examples: 419
• Evaluation examples: 200
• Maximum training length: 3,072 tokens
• Sliding-window stride: 384 tokens
• Training epochs: 2
• Learning rate: 1e-5
• Precision: BF16
• GPU: NVIDIA GeForce RTX 5090
• Fine-tuned by this project: Yes

Performance interpretation

The QASPER-fine-tuned Longformer outperformed both the truncated BERT baseline and the original SQuAD-fine-tuned Longformer checkpoint across Exact Match, token-level F1, evidence recovery, and evidence-token recall.

The model remains imperfect. Exact Match is a strict metric, and QASPER questions frequently contain long, technical, and semantically complex answers. The reported values should not be interpreted as production-level reliability.