Kimi-Audio-7B-SAP-EC

Kimi-Audio-7B-Instruct fine-tuned for dysarthric speech recognition with an extra classifier that predicts the speaker's condition from the audio encoder. The classifier helps training, but its prediction is not given to the text decoder.

Paper: Etiology-Aware Speech Language Models for Dysarthric Speech Recognition (Interspeech 2026) Paper | Code

The four models

All four are Kimi-Audio-7B-Instruct fine-tuned on the same data. They differ only in how the speaker's condition (etiology) is used.

WER in %, lower is better. SemScore higher is better. TORGO is zero-shot (no TORGO training).

Input and output

Input: the prompt Please transcribe the spoken content into written text. and the audio.

Output: the transcript. The file etiology_classifier.pt holds the classifier head (classes: Parkinson's Disease, ALS, Cerebral Palsy, Down Syndrome, Stroke).

Usage

git clone --recursive https://github.com/MorenoLaQuatra/dysarthric-asr
cd dysarthric-asr
pip install -r requirements/kimi.txt -r kimi/third_party/Kimi-Audio/requirements.txt
bash kimi/scripts/setup.sh
source kimi/scripts/env.sh

from kimia_infer.api.kimia import KimiAudio

model = KimiAudio(model_path="morenolq/Kimi-Audio-7B-SAP-EC", load_detokenizer=False)
messages = [
    {"role": "user", "message_type": "text", "content": "Please transcribe the spoken content into written text."},
    {"role": "user", "message_type": "audio", "content": "speech.wav"},
]
_, text = model.generate(messages, output_type="text",
                         text_temperature=0.0, text_top_k=5,
                         audio_temperature=0.8, audio_top_k=10)
print(text)

The classifier head is loaded automatically by kimi/infer.py in the code repository, which saves its prediction as pred_etiology.

For batch inference and the paper evaluation (SAP and TORGO), see the code repository.

Results per condition

Etiology prediction accuracy on SAP: 81.1%.

Training

• Data: Speech Accessibility Project (SAP), release v31-05-2025. 240,047 utterances (547 h) from speakers with ALS, Parkinson's disease, stroke, Down syndrome and cerebral palsy.
• Method: LoRA (rank 16, alpha 32) on the language model and on the Whisper encoder attention. 3 epochs, learning rate 5e-5, global batch 16. Checkpoint chosen by validation loss.
• This repository holds the merged weights (fp32) exactly as evaluated in the paper.

Limitations

• For research on English dysarthric speech recognition.
• The predicted etiology is a helper for transcription, not a diagnosis. Do not use it for clinical decisions.
• Trained on five conditions. Other conditions, languages or recording setups are not tested beyond TORGO.
• No audio or transcripts from SAP are included in this repository.

Citation

If you use this model, please cite the paper:

@inproceedings{laquatra26_interspeech,
  title     = {{Etiology-Aware Speech Language Models for Dysarthric Speech Recognition}},
  author    = {Moreno {La Quatra} and Alkis Koudounas and Valerio Mario Salerno and Sabato Marco Siniscalchi},
  year      = {2026},
  booktitle = {{Interspeech 2026}},
  pages     = {5033--5037},
  doi       = {10.21437/Interspeech.2026-1773},
  issn      = {2958-1796},
}

Acknowledgements

Trained on data from the Speech Accessibility Project (Hasegawa-Johnson et al., 2024). Base model: Kimi-Audio (Ding et al., 2025). License: MIT, as the base model.