T5 Dhivehi Typo Correction

A fine-tuned T5 model for correcting typos in Dhivehi text. This project uses a custom-trained T5-small model to detect and fix spelling errors in Dhivehi text.

Overview

This project implements a spell-checking system using:

• T5-small as the base model
• Custom Dhivehi tokenizer
• Weights & Biases for experiment tracking
• Hugging Face's Transformers library

Training parameters:

• Learning rate: 3e-4
• Batch size: 64
• Training epochs: 3
• Weight decay: 0.01
• Warmup ratio: 0.1
• Maximum sequence length: 128 tokens

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
import torch

# Load the model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("alakxender/dhivehi-quick-spell-check-t5")
model = AutoModelForSeq2SeqLM.from_pretrained("alakxender/dhivehi-quick-spell-check-t5")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

# Correct text
def correct_text(input_text):
    input_text = "fix: " + input_text
    inputs = tokenizer(input_text, return_tensors="pt", max_length=128, truncation=True)
    inputs = inputs.to(device)
    
    outputs = model.generate(
        input_ids=inputs["input_ids"],
        attention_mask=inputs.get("attention_mask", None),
        max_length=128,
        num_beams=4,
        early_stopping=True
    )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)