arxiv-summarization

This model is a fine-tuned version of google/flan-t5-small on a dataset of armanc/scientific_papers (arxiv). It is optimized for summarizing scientific abstracts.

Model Details

• Base Model: google/flan-t5-small
• Training Data: Arxiv Research Papers (article → abstract)
• Fine-Tuned Task: Text Summarization
• Use Case: Generate shorter summaries of long research papers
• License: Apache 2.0

How to Use

from transformers import T5ForConditionalGeneration, T5Tokenizer

model = T5ForConditionalGeneration.from_pretrained("Talina06/arxiv-summarization")
tokenizer = T5Tokenizer.from_pretrained("Talina06/arxiv-summarization")

text = "Summarize: Deep learning is being used to advance medical research, particularly in cancer detection."
inputs = tokenizer(text, return_tensors="pt")
summary_ids = model.generate(**inputs)
summary = tokenizer.decode(summary_ids[0], skip_special_tokens=True)

print("Generated Summary:", summary)

Training Details

• Training Data: 100k+ Arxiv research papers
• Training Framework: Hugging Face Transformers
• Hyperparameters:
  • Learning Rate: 5e-5
  • Batch Size: 8
  • Epochs: 10
• Hardware Used: TPU & GPU

Limitations

• ❌ May struggle with very technical papers (e.g., complex math formulas).

Example Summaries