T5-CNN-25K

A fine-tuned T5-small model for abstractive text summarization, trained on 25,000 examples from the CNN/DailyMail dataset.

Model Description

This model is designed to generate concise abstractive summaries from input text.

It is used as the summarization model in the Text Summarizer application, which supports PDF and text document summarization.

Base Model

• Base model: google-t5/t5-small
• Task: Abstractive text summarization
• Training dataset: CNN/DailyMail
• Training examples: 25,000
• Maximum input length: 512 tokens
• Model format: Safetensors

Intended Use

The model is intended for:

• Summarizing documents and articles
• Generating short, medium, and long summaries
• Educational and research projects
• Demonstrations of transformer-based text summarization

Usage

from transformers import T5Tokenizer, T5ForConditionalGeneration

model_name = "vaibhavvanshu/t5-cnn-25k"

tokenizer = T5Tokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)

text = "summarize: Your text goes here."

inputs = tokenizer(
    text,
    return_tensors="pt",
    max_length=512,
    truncation=True
)

outputs = model.generate(
    **inputs,
    max_length=250,
    num_beams=4,
    early_stopping=True
)

summary = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print(summary)

Training

The model was fine-tuned from T5-small using the CNN/DailyMail summarization dataset.

The project contains multiple experiments using different training dataset sizes. This repository contains the 25K training version selected for the final application.

The final application uses hierarchical summarization for longer documents by:

1. Splitting large documents into manageable chunks.
2. Generating intermediate summaries for each chunk.
3. Combining the intermediate summaries.
4. Performing additional summarization passes when necessary.
5. Producing the final summary according to the requested summary length.

Summary Lengths

The application supports three output lengths:

These settings are implemented by the application rather than being separate models.

Limitations

Like other abstractive summarization models, this model may:

• Omit important information.
• Produce inaccurate statements.
• Occasionally hallucinate information.
• Perform differently depending on the input domain and writing style.

Generated summaries should therefore be reviewed when accuracy is important.

Dataset

The model was trained using the CNN/DailyMail summarization dataset.

Dataset:

cnn_dailymail

The dataset contains news articles paired with human-written reference summaries.

License

This model is released under the Apache License 2.0.

The model is a fine-tuned version of T5-small. Users should also review the licenses and terms associated with the base model and training dataset when using the model.

Related Project

This model is part of the Text Summarizer application, which consists of:

• React frontend
• FastAPI backend
• T5-based summarization model
• PDF text extraction
• OCR support
• Hierarchical document summarization

The trained model is hosted on Hugging Face and the application can use this repository for model inference.