whisper-small-he-acft
Hebrew Whisper Small with ACFT (Audio-Context Fine-Tuning) for optimized short-audio performance, compatible with FUTO Keyboard and whisper.cpp.
Training
Two-stage pipeline:
- Fine-tune: openai/whisper-small on ivrit-ai/whisper-training (~400h Hebrew) → amitkot/whisper-small-he
- ACFT: Fine-tuned model on google/fleurs (he_il) using FUTO-aligned ACFT (partial encoder with truncated positional embeddings, 8 epochs, batch_size=1)
- Hardware: Apple M4 (MPS)
- Method: Distillation-based — teaches model to handle short audio contexts without repeating
Usage
from transformers import WhisperProcessor, WhisperForConditionalGeneration
processor = WhisperProcessor.from_pretrained("amitkot/whisper-small-he-acft")
model = WhisperForConditionalGeneration.from_pretrained("amitkot/whisper-small-he-acft")
For FUTO Keyboard / whisper.cpp, convert to ggml:
uv run python scripts/pipeline.py \
--finetune-config configs/hebrew_small_finetune.yaml \
--config configs/hebrew_small_acft.yaml
Training pipeline
Trained using whisper-acft-pipeline.
See also
- amitkot/whisper-small-he — Base fine-tuned model (before ACFT)
- amitkot/whisper-tiny-he-acft — Smaller/faster variant
- FUTO whisper-acft — ACFT method reference