Kimi-Audio-7B-SAP-SFT

Kimi-Audio-7B-Instruct fine-tuned for dysarthric speech recognition with the standard transcription objective. This is the baseline of the paper: it does not use any information about the speaker's condition.

Paper: Etiology-Aware Speech Language Models for Dysarthric Speech Recognition (Interspeech 2026) Paper | Code

The four models

All four are Kimi-Audio-7B-Instruct fine-tuned on the same data. They differ only in how the speaker's condition (etiology) is used.

Model	Strategy	SAP WER	SAP SemScore	TORGO WER
Kimi-Audio-7B-SAP-SFT	SFT: standard fine-tuning	8.30	90.48	18.0
Kimi-Audio-7B-SAP-EC	EC: etiology classification	8.49	90.42	16.7
Kimi-Audio-7B-SAP-EH	EH: etiology hinting	8.22	90.63	16.2
Kimi-Audio-7B-SAP-EP	EP: etiology prediction (best)	7.77	91.05	15.8

WER in %, lower is better. SemScore higher is better. TORGO is zero-shot (no TORGO training).

Input and output

Input: the prompt Please transcribe the spoken content into written text. and the audio.

Output: the transcript (lowercase).

Usage

git clone --recursive https://github.com/MorenoLaQuatra/dysarthric-asr
cd dysarthric-asr
pip install -r requirements/kimi.txt -r kimi/third_party/Kimi-Audio/requirements.txt
bash kimi/scripts/setup.sh
source kimi/scripts/env.sh

from kimia_infer.api.kimia import KimiAudio

model = KimiAudio(model_path="morenolq/Kimi-Audio-7B-SAP-SFT", load_detokenizer=False)
messages = [
    {"role": "user", "message_type": "text", "content": "Please transcribe the spoken content into written text."},
    {"role": "user", "message_type": "audio", "content": "speech.wav"},
]
_, text = model.generate(messages, output_type="text",
                         text_temperature=0.0, text_top_k=5,
                         audio_temperature=0.8, audio_top_k=10)
print(text)

For batch inference and the paper evaluation (SAP and TORGO), see the code repository.

Results per condition

	ALS	Parkinson's Disease	Stroke	Down Syndrome	Cerebral Palsy	Overall
SAP WER	5.98	6.46	10.87	18.29	14.91	8.30
SAP SemScore	91.63	94.07	91.93	79.86	83.89	90.48

TORGO (zero-shot)	Healthy words	Healthy sentences	Dysarthric words	Dysarthric sentences	Overall
WER / SemScore	22.9 / 84.5	2.4 / 98.3	57.2 / 59.5	28.6 / 72.9	18.0 / 79.5

Training

Data: Speech Accessibility Project (SAP), release v31-05-2025. 240,047 utterances (547 h) from speakers with ALS, Parkinson's disease, stroke, Down syndrome and cerebral palsy.
Method: LoRA (rank 16, alpha 32) on the language model and on the Whisper encoder attention. 3 epochs, learning rate 5e-5, global batch 64. Checkpoint chosen by validation loss.
This repository holds the merged weights (fp32) exactly as evaluated in the paper.

Limitations

For research on English dysarthric speech recognition.
The predicted etiology is a helper for transcription, not a diagnosis. Do not use it for clinical decisions.
Trained on five conditions. Other conditions, languages or recording setups are not tested beyond TORGO.
No audio or transcripts from SAP are included in this repository.

Citation

If you use this model, please cite the paper:

@inproceedings{laquatra26_interspeech,
  title     = {{Etiology-Aware Speech Language Models for Dysarthric Speech Recognition}},
  author    = {Moreno {La Quatra} and Alkis Koudounas and Valerio Mario Salerno and Sabato Marco Siniscalchi},
  year      = {2026},
  booktitle = {{Interspeech 2026}},
  pages     = {5033--5037},
  doi       = {10.21437/Interspeech.2026-1773},
  issn      = {2958-1796},
}

Acknowledgements

Trained on data from the Speech Accessibility Project (Hasegawa-Johnson et al., 2024). Base model: Kimi-Audio (Ding et al., 2025). License: MIT, as the base model.

morenolq/Kimi-Audio-7B-SAP-SFT