Moonshine Tiny (27M) for dysarthric speech: SFT + RL (SAPC2)

Fine-tuned for the Speech Accessibility Project Challenge 2 (SAPC2): English speech from speakers with ALS, cerebral palsy, Down syndrome, Parkinson's disease and stroke. Starting point: moonshine-ai/moonshine-tiny, fine-tuned on SAPC2 Train (SOAP, duration-balanced sampling), then one epoch of RLOO-style RL with a WER/CER reward and a cross-entropy anchor. This is the final RL checkpoint (step 4446), used for the Track 2 submission.

Evaluation

The checkpoint was selected on a fixed 1,000-recording subset of the official Dev set; the scores below are on the full Dev set (47,929 recordings). Scores use the competition normalizer against the better of the two references; they are within about 0.1 points of the official SCTK scorer on the same predictions.

Full Dev (47,929 recordings)

Whole-recording decoding (200 s windows). Quick scorer: competition normalizer, better of the two references. Official: the challenge's evaluate.py with sclite against both references (where it was run; it scores about 0.05 points lower than the quick scorer).

Exact transcripts 61.47%; substitutions 13.32%, deletions 2.99%, insertions 4.00% of reference words (uncapped WER 20.32%).

CPU streaming package cost

The ONNX package run through the challenge's streaming interface on a 300-recording Dev subset (0.64 h of audio), two worker processes, one ONNX Runtime thread each, on a laptop CPU, with no other load. Compute is the wall-clock seconds spent per hour of audio; WER/CER are on that subset only.

WER by duration

WER by etiology

Runaway transcripts (hypothesis over 1.5x the reference length): 355, 8.98% of all errors; truncated transcripts: 210.

CPU streaming package (Track 2)

The submission packages this checkpoint as ONNX FP32 (encoder, decoder and decoder-with-cache graphs exported with optimum; 240 MB zipped) behind the challenge's 5-method streaming interface: greedy partial transcripts every 0.5 s for the first 4 s and every 10 s afterwards, then one final decode of the whole recording with the configured beam width. ONNX and PyTorch greedy transcripts match on 39 of 40 parity recordings; through the streaming interface the package scores 18.75% greedy / 17.36% beam 4 on a 300-recording Dev pilot.

Streaming latency (CPU package)

Measured with the challenge's own local decoder and latency script on a 100-recording Dev subset (12.5 min of audio), audio sent at real-time pace in 100 ms chunks, one ONNX Runtime thread, on a laptop CPU. TTFT here is the first partial relative to the start of the audio (the official metric subtracts the MFA speech onset, which this subset does not carry); partial transcripts are greedy at every setting, so TTFT does not depend on the final beam width. TTLT is the final transcript after the last audio chunk.

Training recipe

{
  "objective": "rl",
  "init": "runs/sft_continue_v1/best",
  "peak_lr": 2e-06,
  "epochs": 1,
  "balanced": true,
  "balance_max_repeats": 2,
  "rl": {
    "samples_per_recording": 2,
    "kl_weight": 0.02,
    "ce_anchor": 0.1
  },
  "policy": {
    "max_items": 96,
    "padded_seconds": 576,
    "attention_cost": 60000,
    "decoder_tokens": 12288,
    "accumulation": 2
  },
  "seed": 20261023
}

Optimizer: blocked SOAP (Adam in a Shampoo eigenbasis), FP32 weights with BF16 autocast. Code: the moonshine package in the project repository (training, distillation, RL and evaluation), with resumable checkpoints.

Files

Hugging Face format (config.json, model.safetensors, tokenizer/processor files). Load with transformers (MoonshineForConditionalGeneration or CohereAsrForConditionalGeneration).