Whisper Small - Korean to English Audio Translator

This repository contains a fine-tuned version of openai/whisper-small optimized for translating spoken Korean audio directly into English text transcripts.

The model was adapted using Parameter-Efficient Fine-Tuning (PEFT) via 4-bit QLoRA targeting custom media broadcasts.

Model Description

• Developed by: didiudom94
• Model type: Whisper (Sequence-to-Sequence Audio Transformer)
• Language(s) (ISOs): Korean (ko) to English (en)
• License: Apache 2.0
• Finetuned from model: openai/whisper-small

Training Hyperparameters & Infrastructure

• Hardware: NVIDIA A100 GPU
• Quantization: 4-bit NormalFloat (nf4) with double quantization
• LoRA Configurations: Rank ((r)) = 32, Alpha ((\alpha)) = 64
• Learning Rate: 1e-4
• Precision: Mixed Precision (bf16)
• Optimizer updates: Direct weights optimization (Batch Size = 32, Gradient Accumulation = 1)

How to Load and Use

You can easily reload this model for inference using the code snippet below:

import torch
from transformers import WhisperForConditionalGeneration, WhisperProcessor
from peft import PeftModel, PeftConfig

model_id = "openai/whisper-small"
peft_model_id = "didiudom94/whisper-small-ko-to-en-translator"

# Load unified processor
processor = WhisperProcessor.from_pretrained(peft_model_id)

# Load base architecture in low-precision
base_model = WhisperForConditionalGeneration.from_pretrained(
    model_id,
    load_in_4bit=True,
    device_map="auto"
)

# Merge fine-tuned LoRA weights
model = PeftModel.from_pretrained(base_model, peft_model_id)