This model forecasts whether developers refactor a Java production file before the next release. It reads three views of the file, all written before the release: its code (the file's skeleton and its methods), its history (each commit that changed it: the message and the file's changed lines) and its discourse (the bodies, comments and reviews of the issues and pull requests linked to those commits). It returns a probability.
It is the model of A Language Model That Reads Code, History and Discourse, Chapter 5 of Binamra Aryal's MS thesis at The University of Texas at Arlington.
| Training data | 10.5281/zenodo.24117444 (version 2.1), the training releases of five Java repos |
| Code and notebooks | refactoring-forecast-replication |
| Mined inputs | 10.5281/zenodo.24182007 |
| Part | What it is |
|---|---|
| Code encoder | microsoft/codebert-base, the first token's vector of each window |
| History encoder | a second copy of microsoft/codebert-base, the first token's vector |
| Discourse encoder | sentence-transformers/all-mpnet-base-v2, the mean of the token vectors |
| Windows | 510 tokens without overlap (code, history); 382 tokens, each starting halfway through the last (discourse); at most 64 windows per text |
| Text budget | each view of a row reads at most 64 windows: the skeleton first, then the methods largest first; the newest commits; the newest discourse texts |
| Per text | the mean of its window vectors |
| Per view | attention pooling over the view's texts; a learned "empty" vector when a row has no text in a view |
| Output | the three view vectors joined (2,304 numbers), dropout 0.2, one linear layer, sigmoid |
All weights, the encoders' included, were fine-tuned together on the labels: AdamW, learning rate 2e-5 (encoders) and 1e-3 (new layers), 16 rows per step, bfloat16, on one A100. Each epoch used every refactored training row and as many unrefactored rows of the same repo, drawn anew. Training stopped after 7 epochs; these are the weights of epoch 4, the epoch with the highest validation AP (0.106).
On the test releases (200,438 rows, 3.2% refactored), at the decision threshold of 0.794 chosen on the validation releases:
| Precision | Recall | F1 | ROC-AUC | AP |
|---|---|---|---|---|
| 0.095 | 0.297 | 0.144 | 0.713 | 0.079 |
It ranks refactored files above the rest better than chance in every repo, with an AP 2.0 to 3.4 times the repo's base rate, but only about one flag in ten is right. Use it to order files for review, not to decide alone.
The weights fit the model classes of the replication package's Notebook 4. forecaster.py holds
those classes and a small loader:
# pip install torch transformers safetensors huggingface_hub
from huggingface_hub import snapshot_download
import sys
folder = snapshot_download("Binamra00/refactoring-forecaster")
sys.path.insert(0, folder)
from forecaster import load
model = load(folder) # downloads the two base tokenizers and configs on first use
p = model.probability(code=[skeleton, *methods], history=commit_texts, discourse=discourse_texts)
print(p, p >= model.threshold)
Build the texts the way the dataset does (Notebook 3 of the replication package): the skeleton is
the file with every method body replaced by { … } and the package line, imports and comments
removed; each method is one text with its Javadoc; a history text is
<COMMIT_MESSAGE>the message</COMMIT_MESSAGE>
<FILE>the file's path in that commit</FILE>
<REMOVED>removed lines</REMOVED>
<ADDED>added lines</ADDED>
leaving out the <REMOVED> or <ADDED> part when it has no lines. History and discourse texts go
newest first, and none may be dated at or after the release.
| File | Holds |
|---|---|
model.safetensors |
the trained weights (1.4 GB, float32) |
config.json |
the windows, the text budget, the threshold and the test scores |
forecaster.py |
the model classes and load() |
The weights are released under Apache-2.0. They are fine-tuned from microsoft/codebert-base (MIT)
and sentence-transformers/all-mpnet-base-v2 (Apache-2.0).
@misc{aryal2026forecaster,
author = {Aryal, Binamra},
title = {Refactoring forecaster (code, history and discourse)},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/Binamra00/refactoring-forecaster}}
}
@dataset{aryal2026data,
author = {Aryal, Binamra},
title = {Code, Commit History and Issue/PR Discourse for Forecasting Refactoring},
year = {2026},
version = {2.1},
publisher = {Zenodo},
doi = {10.5281/zenodo.24117444}
}
The model fine-tunes CodeBERT (Feng et al., 2020) and all-mpnet-base-v2, a Sentence-BERT model (Reimers and Gurevych, 2019) built on MPNet (Song et al., 2020). Its labels come from RefactoringMiner (Tsantalis et al., 2022). It was trained with Hugging Face Transformers and PyTorch on a RunPod A100.
microsoft/codebert-base