Encoder: DistilBERT: IMDb sentiment

Fine-tuned for Sultan0336's encoder/decoder comparison assignment. Base model: distilbert/distilbert-base-uncased. Two-class sequence classification: 0 NEGATIVE, 1 POSITIVE. DistilBERT is bidirectional; DistilGPT2 uses causal attention with a classification head. This checkpoint predicts sentiment; it is not a conversational generator.

Data and training

IMDb: 4,000 training reviews, 500 validation reviews, and 1,000 held-out test reviews. Each split is class-balanced. Validation comes from the original training split; test comes from the original test split. Exact normalized duplicates are removed. HTML line breaks are removed and whitespace is normalized. Maximum length: 128 tokens. Seed 42, 2 epochs, AdamW, learning rate 0.00002, weight decay 0.01, warmup ratio 0.1, batch size 8, accumulation 2. Best validation macro-F1 selects the checkpoint.

Measured held-out results

{
  "test_loss": 0.39062610268592834,
  "test_accuracy": 0.838,
  "test_macro_precision": 0.8381352541016407,
  "test_macro_recall": 0.838,
  "test_macro_f1": 0.8379837983798379,
  "test_runtime": 1.156,
  "test_samples_per_second": 865.045,
  "test_steps_per_second": 54.498,
  "training_seconds": 51.23652573800001
}

Limitations

Small training sample; one seed; movie-review domain; no neutral label. Long reviews are truncated. Sarcasm and mixed sentiment can be misclassified. Softmax scores are not calibrated probabilities. This comparison cannot isolate architecture from pretraining, tokenizer, and parameter-count differences. Dataset terms: https://huggingface.co/datasets/stanfordnlp/imdb

Load

from transformers import pipeline
classifier = pipeline("text-classification", model="Sultan0336/distilbert-imdb-sentiment")
print(classifier("An enjoyable movie.", truncation=True, max_length=128))