Movie Review Generator - GPT-2

A fine-tuned GPT-2 decoder-only language model that generates movie-review-style text based on patterns learned from the IMDb movie review dataset.

The model uses causal language modeling, meaning it predicts the next token based on the preceding tokens to generate coherent sequences of text.

Model Details

Intended Use

This model is intended for:

• Generating movie-review-style text.
• Educational experiments with generative language models.
• Exploring decoder-only Transformer architectures.
• Demonstrating text generation using fine-tuned GPT-2.

Example prompt:

Movie review:

The model continues the prompt by generating movie-review-style text.

This model is a text generator, not a sentiment classifier. It does not guarantee positive or negative sentiment in its generated reviews.

Dataset

The model was fine-tuned using the IMDb Movie Reviews Dataset.

The dataset contains movie reviews with positive and negative sentiment labels.

For this project, only the review text was used. The original sentiment labels were discarded and were not included in the training objective.

Data Preparation

The following preprocessing steps were performed:

1. Removed HTML tags.
2. Decoded HTML entities.
3. Removed URLs.
4. Normalized extra whitespace.
5. Added the Movie review: prompt prefix.
6. Added the GPT-2 end-of-sequence token.
7. Tokenized the text using the GPT-2 tokenizer.

Dataset Split

The notebook uses a selected subset of the IMDb dataset.

Training Configuration

Evaluation

The model was evaluated on the held-out evaluation subset using the causal language modeling objective.

The notebook calculates:

• Evaluation Loss: Measures the model's next-token prediction error.
• Perplexity: Calculated as the exponential of evaluation loss.

Lower loss and perplexity generally indicate better next-token prediction performance. These metrics do not directly measure human-rated review quality, factual accuracy, or grammatical correctness.

How to Use the Model

Anyone can load this model directly from Hugging Face using the Transformers library.

1. Install Dependencies

pip install transformers torch

2. Load the Model and Generate a Review

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

# Hugging Face model repository
model_id = "ullahhafeez321/movie-review-generator-gpt2"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Load fine-tuned GPT-2 model
model = AutoModelForCausalLM.from_pretrained(model_id)

# Use GPU if available
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()

# Prepare input prompt
prompt = "Movie review:"

inputs = tokenizer(prompt, return_tensors="pt").to(device)

# Generate movie-review-style text
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=150,
        do_sample=True,
        temperature=0.8,
        top_k=50,
        top_p=0.92,
        repetition_penalty=1.2,
        no_repeat_ngram_size=3,
        pad_token_id=tokenizer.eos_token_id
    )

# Decode generated text
generated_review = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print(generated_review)

3. Use a Custom Prompt

You can also provide your own movie-review starting text:

prompt = "Movie review: This movie tells an interesting story"

inputs = tokenizer(prompt, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=150,
        do_sample=True,
        temperature=0.8,
        top_p=0.92,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• The model is fine-tuned on a selected subset of IMDb reviews, not the entire dataset.
• Generated text may contain repetition, grammatical errors, or irrelevant statements.
• It may reproduce biases and patterns present in the training data.
• It does not have real-world movie knowledge or guarantee factual information.
• It is not instruction-tuned and is primarily designed for continuation from a movie-review prompt.
• It does not perform sentiment classification.

Project Information

This model was developed as an educational project to demonstrate fine-tuning and text generation using a pretrained decoder-only GPT-2 language model.

Hugging Face Repository:

ullahhafeez321/movie-review-generator-gpt2

Base Model: openai-community/gpt2

Dataset: stanfordnlp/imdb

---

Built with Hugging Face Transformers and PyTorch.