Decoder: DistilGPT2: IMDb sentiment

Fine-tuned for Sultan0336's encoder/decoder comparison assignment. Base model: distilbert/distilgpt2. Two-class sequence classification: 0 NEGATIVE, 1 POSITIVE. DistilBERT is bidirectional; DistilGPT2 uses causal attention with a classification head. This checkpoint predicts sentiment; it is not a conversational generator.

Data and training

IMDb: 4,000 training reviews, 500 validation reviews, and 1,000 held-out test reviews. Each split is class-balanced. Validation comes from the original training split; test comes from the original test split. Exact normalized duplicates are removed. HTML line breaks are removed and whitespace is normalized. Maximum length: 128 tokens. Seed 42, 2 epochs, AdamW, learning rate 0.00002, weight decay 0.01, warmup ratio 0.1, batch size 8, accumulation 2. Best validation macro-F1 selects the checkpoint.

Measured held-out results

{
  "test_loss": 0.3785669207572937,
  "test_accuracy": 0.839,
  "test_macro_precision": 0.8391098715983979,
  "test_macro_recall": 0.839,
  "test_macro_f1": 0.8389869579435933,
  "test_runtime": 1.9742,
  "test_samples_per_second": 506.536,
  "test_steps_per_second": 31.912,
  "training_seconds": 153.86052593199997
}

Limitations

Small training sample; one seed; movie-review domain; no neutral label. Long reviews are truncated. Sarcasm and mixed sentiment can be misclassified. Softmax scores are not calibrated probabilities. This comparison cannot isolate architecture from pretraining, tokenizer, and parameter-count differences. Dataset terms: https://huggingface.co/datasets/stanfordnlp/imdb

Load

from transformers import pipeline
classifier = pipeline("text-classification", model="Sultan0336/distilgpt2-imdb-sentiment")
print(classifier("An enjoyable movie.", truncation=True, max_length=128))