Product Description Generator

Overview

This repository contains a fine-tuned model for generating high-quality product descriptions. The model is based on the t5-base and has 223 million parameters. It has been fine-tuned on the Amazon Product Dataset, which contains 10 million examples, with the cleaned version having 0.5 million examples.

Usage

T5 model expects a task related prefix: since it is a description generation task, we will add a prefix "description: "

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
tokenizer = AutoTokenizer.from_pretrained("Ateeqq/product-description-generator")
model = AutoModelForSeq2SeqLM.from_pretrained("Ateeqq/product-description-generator").to(device)

def generate_description(title):
    input_ids = tokenizer(f'description: {title}', return_tensors="pt", padding="longest", truncation=True, max_length=128).input_ids.to(device)
    outputs = model.generate(
        input_ids,
        num_beams=5,
        num_beam_groups=5,
        num_return_sequences=5,
        repetition_penalty=10.0,
        diversity_penalty=3.0,
        no_repeat_ngram_size=2,
        temperature=0.7,
        max_length=128
    )
    return tokenizer.batch_decode(outputs, skip_special_tokens=True)

title = '18-Piece Kitchen Dinnerware white Set, Plates, Dishes, Bowls'
generate_description(title)

Outputs:

['18-Piece Kitchen Dinnerware White Set, Plates, Dishes and Bowls. This set includes a large bowl with two plates for serving dinner or dessert dishes in the kitchen. The white plate is made of durable stainless steel that will not fade over time.',
 'The 18 Piece Kitchen Dinnerware Set features a white plate, dish and bowl. This set is made of durable stainless steel with an elegant design that will add elegance to your kitchen.',
 'This 18-piece dinnerware set is made of durable stainless steel. It features a white finish and comes with an easy to clean handle for ease of cleaning. The bowls are dishwasher safe, microwave safe or can be used as tablecloths in the kitchen.',
 'Kitchen Dinnerware is a great addition to your kitchen. This 18-piece set includes four plates, two dishes and three bowls for serving food or beverages in the dining room with an elegant design that will add sophistication to any tabletop setting.',
 "18-Piece Kitchen Dinnerware White Set, Plates & DisheSize(LxW) Size (HXD), Material is made of high quality stainless steel and the bowl has an elegant design. The dish set includes 1 plate with 2 plates; 3 dishes for 4 cups or 6 serving spoon/bowl 2. This dinner table sets are perfect to serve your food in style while also being easy on clean up 3. These white kitchen dining tables have been designed by professional chefs using only premium materials that will not scratch surfaces 4. Easy cleaning dishwasher safe 5. Durable material can be used as disposable wash cloth 6. Perfect gift idea For family members who love cooking."]

Features

• Architecture: t5-base (223M parameters)
• Training Dataset: Trained on 0.5 million cleaned examples
• Training Time:
  • Hardware: Colab T4 GPU
  • Speed: 4.91 iterations/second
  • Training Time: 5:53:49
  • Metrics:
    • Loss: 2.53
    • Training Loss Step: 0.095
    • Validation Loss Step: 1.670
    • Validation Loss Epoch: 2.290
    • Training Loss Epoch: 1.410

Data Preparation

• Training Data: First 250,000 examples for train
• Validation Data: First 40,000 examples for validation
• Source Max Token Length: 50
• Target Max Token Length: 300
• Batch Size: 1
• Max Epochs: 1

Future Work

• Update Training Data: Retrain the model using the latest 0.5 million cleaned examples.
• Optimize Training Parameters: Experiment with different batch sizes, learning rates, and epochs to further improve model performance.
• Expand Dataset: Incorporate more diverse product datasets to enhance the model's versatility and robustness.