whisper-small-fo-fr (Malinali)

Fine-tuned openai/whisper-small for Fon (Fongbe) speech → French text (speech translation, not Fon ASR).

Prototype. Gated on pragmatic metrics (sentence embedding similarity + chrF) on held-out street-style clips. Not medical/legal grade. Expect misses on noise, codeswitching, and long utterances.

Credits (required)

• Speech data: GbeBenin/FFSTC-2 (FFSTC 2: Extending the Fongbe to French Speech Translation Corpus).
• Base model: OpenAI Whisper (openai/whisper-small, Apache-2.0).

Intended use

Decode with French forced decoder / language=fr, task=transcribe (Whisper has no <|fon|> token; training used this setup).

Suggested generation kwargs (anti-hallucination):

dict(
    language="fr",
    task="transcribe",
    num_beams=4,
    length_penalty=0.8,
    repetition_penalty=1.15,
    no_repeat_ngram_size=3,
    max_new_tokens=96,
)

Training (summary)

• Filtered FFSTC-2 French targets (drop address/parliament; upsample street).
• Marian FR→EN→FR simplification when meaning preserved (sem_sim ≥ 80).
• SpecAugment, cosine LR 2e-5, best checkpoint by MiniLM semantic similarity.
• Early-stopped ~epoch 2 (best step 4400).

On-device pack

Sherpa-onnx int8 graphs: malinali-app/whisper-small-fo-fr-onnx.

Limitations

• Literary / parliamentary FFSTC-2 style was down-weighted; street meaning preferred.
• Gender/pronoun swaps and occasional topic drift still occur.
• Single-speaker studio bias from the source corpus.

Citation

Please cite FFSTC-2 and Whisper if you use this model.