Step-Audio 2 Mini Luganda-English Bidirectional S2ST LoRA
This repository contains a LoRA adapter for stepfun-ai/Step-Audio-2-mini trained
with direction prompts for both Luganda-to-English and English-to-Luganda
speech-to-speech translation.
Adapter repository: yigagilbert/stepaudio2-mini-luganda-english-bidirectional-s2st-lora
Intended Use
Research and development for bidirectional Luganda-English speech translation. Validate with native speakers before production use.
Training Data
The adapter was trained from yigagilbert/luganda-english-cleaned-v1-split by expanding
each aligned pair into two supervised examples:
- Luganda speech input -> English text and English speech tokens
- English speech input -> Luganda text and Luganda speech tokens
Each example uses a direction-specific system prompt so that one adapter can learn both translation directions.
Evaluation
The checkpoint uploaded here was evaluated on 200 held-out validation examples per direction. Text metrics are computed on the generated text channel; audio samples are synthesized from generated Step-Audio tokens with the upstream Token2wav decoder.
Text Metrics
| Direction | System | Count | BLEU higher | chrF higher | WER lower | COMET higher |
|---|---|---|---|---|---|---|
| Luganda -> English | Focused one-direction LoRA | 200 | 32.530 | 54.535 | 0.574 | 0.717 |
| Luganda -> English | This bidirectional LoRA | 200 | 19.032 | 42.447 | 0.737 | 0.640 |
| English -> Luganda | Focused one-direction LoRA control | 200 | 0.120 | 14.050 | 1.300 | - |
| English -> Luganda | This bidirectional LoRA | 200 | 11.155 | 45.331 | 0.811 | 0.678 |
| English -> Luganda | ASR + MT cascade baseline | 200 | 4.026 | 34.734 | 2.039 | 0.624 |
Speech Metrics
Speech metrics were computed on 20 audio-aligned samples per direction using WavLM-large SpeechBERTScore-style similarity and MFCC+DTW MCD. These should be treated as supporting audio-channel diagnostics, not as standalone translation-quality measures.
| Direction | Count | SpeechBERT P higher | SpeechBERT R higher | SpeechBERT F1 higher | MCD lower |
|---|---|---|---|---|---|
| Luganda -> English | 20 | 0.560 | 0.582 | 0.570 | 639.360 |
| English -> Luganda | 20 | 0.502 | 0.572 | 0.523 | 614.130 |
Retention and Composition Probes
The bidirectional adapter does not recover the base model's unrelated multilingual speech translation ability on small FLEURS probes: Spanish -> English BLEU is 0.293 and Japanese -> English BLEU is 0.163 on 20 samples each. However, it does recover target-script compliance on Luganda -> Chinese and Luganda -> Japanese prompt probes (100% target-script hit rate in both cases). The semantic quality of those composition outputs is poor, with roundtrip BLEU below 1, so these probes should not be read as successful Luganda -> Chinese/Japanese translation.
Compared with the earlier Luganda-to-English-only LoRA adapter, this bidirectional adapter adds English-to-Luganda capability, but Luganda-to-English quality is lower than the specialized one-direction adapter. The bidirectional run was also shorter (9,000 steps) than the focused adapter run, so the gap combines multi-task interference and training duration.
Loading
Load this as a PEFT adapter on top of stepfun-ai/Step-Audio-2-mini.
Limitations
- Automatic metrics for Luganda can understate useful translations because of spelling and morphology variation.
- Speech quality should be checked by listening to synthesized samples.
- This adapter is a research artifact and should not be used for high-stakes translation without human validation.
License
Adapter code and metadata are Apache-2.0. Check the dataset and base model licenses separately before redistribution.