Moonshine student m24, SAPC2 compression ladder (epoch 2 with KD from Tiny RL, then RLOO)

Fine-tuned for the Speech Accessibility Project Challenge 2 (SAPC2): English speech from speakers with ALS, cerebral palsy, Down syndrome, Parkinson's disease and stroke.

Evaluation

The checkpoint was selected on a fixed 1,000-recording subset of the official Dev set; the scores below are on the full Dev set (47,929 recordings). Scores use the competition normalizer against the better of the two references; they are within about 0.1 points of the official SCTK scorer on the same predictions.

Full Dev (47,929 recordings)

Whole-recording decoding (200 s windows). Quick scorer: competition normalizer, better of the two references. Official: the challenge's evaluate.py with sclite against both references (where it was run; it scores about 0.05 points lower than the quick scorer).

decoding	WER	CER	official SCTK WER	official SCTK CER	empty transcripts	capped (ran to the token limit)
greedy	21.87%	15.11%	21.82%	15.07%	0	76
beam 4	19.74%	13.66%	19.69%	13.62%	0	125

Exact transcripts 56.79%; substitutions 14.93%, deletions 4.17%, insertions 5.18% of reference words (uncapped WER 24.28%).

CPU streaming package cost

The ONNX package run through the challenge's streaming interface on a 300-recording Dev subset (0.64 h of audio), two worker processes, one ONNX Runtime thread each, on a laptop CPU, with no other load. Compute is the wall-clock seconds spent per hour of audio; WER/CER are on that subset only.

final decode	compute per audio hour	WER	CER
greedy	298 s	23.35%	15.60%
beam 4	356 s	20.93%	13.60%

WER by duration

bucket	recordings	WER	share of all errors
1-2s	670	28.12%	0.41%
2-3s	4354	16.60%	2.46%
3-5s	14700	13.93%	9.59%
5-8s	14739	17.45%	17.14%
8-12s	7023	23.07%	14.99%
12-20s	3867	27.28%	16.68%
20-35s	1657	26.27%	15.37%
>35s	919	27.55%	23.35%

WER by etiology

etiology	recordings	WER	share of all errors
ALS	10232	18.12%	18.25%
Cerebral Palsy	11472	33.39%	28.36%
Down Syndrome	7719	27.86%	14.47%
Parkinson's Disease	14543	17.23%	33.06%
Stroke	3963	21.17%	5.87%

Runaway transcripts (hypothesis over 1.5x the reference length): 339, 12.00% of all errors; truncated transcripts: 296.

Streaming latency (CPU package)

Measured with the challenge's own local decoder and latency script on a 100-recording Dev subset (12.5 min of audio), audio sent at real-time pace in 100 ms chunks, one ONNX Runtime thread, on a laptop CPU. TTFT here is the first partial relative to the start of the audio (the official metric subtracts the MFA speech onset, which this subset does not carry); partial transcripts are greedy at every setting, so TTFT does not depend on the final beam width. TTLT is the final transcript after the last audio chunk.

final beams	TTFT median	TTLT median	TTLT p90	TTLT p95	TTLT max	full-Dev WER / CER
1	419 ms	141 ms	273 ms	329 ms	1806 ms	21.87% / 15.11%
4	419 ms	171 ms	378 ms	475 ms	3648 ms	19.74% / 13.66%

Training recipe

{
  "objective": "rl",
  "init": "/runs/sapc_moonshine/runs/m24_e2/best",
  "kd_weight": 0.0,
  "peak_lr": 2e-06,
  "epochs": 1,
  "balanced": true,
  "balance_max_repeats": 2,
  "rl": {
    "samples_per_recording": 4,
    "kl_weight": 0.02,
    "ce_anchor": 0.1,
    "logprob_reduction": "sum"
  },
  "policy": {
    "max_items": 96,
    "padded_seconds": 576,
    "attention_cost": 60000,
    "decoder_tokens": 12288,
    "accumulation": 2
  },
  "seed": 20261027
}

Optimizer: blocked SOAP (Adam in a Shampoo eigenbasis), FP32 weights with BF16 autocast. Code: the moonshine package in the project repository (training, distillation, RL and evaluation), with resumable checkpoints.

Files

Hugging Face format (config.json, model.safetensors, tokenizer/processor files). Load with transformers (MoonshineForConditionalGeneration or CohereAsrForConditionalGeneration).

vanshnawander/sapc2-moonshine-m24