FLAN-T5 Small Fine-Tuned on Gigaword for Headline Generation

This model is a fine-tuned checkpoint of google/flan-t5-small for turning short news text into concise headline-style output.

Training Setup

• Dataset: SalmanFaroz/gigaword (article -> summary)
• Input format: headline: <article>
• Output format: <headline>
• Train samples: 4000
• Validation samples: 300
• Max steps: 70
• Training loss: 2.465707833426339

Usage (Transformers)

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = "vtayyab6/flan-t5-small-gigaword-headline-ft"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

text = "The government announced a new transport reform plan on Tuesday to improve rail connectivity across the country."
prompt = f"headline: {text}"
inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=192)
outputs = model.generate(**inputs, max_new_tokens=20, num_beams=4, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Sample Predictions

Example 1

Input: in a blow to his attorney 's efforts to win a reduced prison sentence , a court-ordered brain scan revealed no abnormalities that might explain the behavior of lionel tate , once the youngest person in modern u.s. his...

Target: tate brain scan shows no abnormalities

Prediction: lionel tate 's brain scan shows no abnormalities

Example 2

Input: the eldest daughter of former chilean dictator augusto pinochet was arrested at a washington airport after fleeing chile where she has been charged with tax fraud , a government spokesman said wednesday .

Target: pinochet 's daughter arrested in washington

Prediction: eldest daughter of chilean dictator arrested at washington airport

Example 3

Input: a rights group said thursday that bangladesh should be expelled from the u.n. human rights council and be banned from taking part in peacekeeping missions until it addresses abuses by the nation 's police and security...

Target: rights group urges expulsion of bangladesh from un human rights council

Prediction: bangladesh should be expelled from u.n. human rights council

Notes

• This checkpoint was trained as a fast portfolio demo using a subsample.
• For stronger quality, train with larger sample size and more steps.