This repository contains the Core ForesightLM checkpoint based on distilgpt2.
ForesightLM studies whether a token-level autoregressive language model can acquire sentence-level foresight through an auxiliary sentence-boundary future semantic objective. The model preserves standard next-token generation while adding a learned projection head for future sentence embedding prediction.
distilgpt2sentence-transformers/all-MiniLM-L6-v2lambda_future = 0.08tau = 0.07This checkpoint is intended for research on:
This model is a small research prototype. It should not be treated as a production-quality text generator.
Automatic metrics show that semantic reranking is a strong component by itself. Foresight training improves several diagnostics but does not uniformly dominate a reranked baseline. Direct future-head reranking exposes a calibration gap.
Human evaluation protocol files are released in the GitHub repository, but human judgments are still being collected and will be added in a later revision.
Code, SLURM scripts, evaluation summaries, compute-cost accounting, bootstrap confidence intervals, qualitative examples, and reproducibility manifests are available at:
https://github.com/Ahmet2001/foresightLM
Large generation JSONL files and training data are not included in this model repository.
If you use this checkpoint, please cite the ForesightLM project repository until a paper DOI/arXiv identifier is available.
model.safetensors 328 MBfuture_head.pt 4 MBdistilbert/distilgpt2