LvW-Dictate-1.5B
Speech in, clean text out: Phonon-2.5 transcribes, a 0.9B cleanup model removes the ums, repeats and false starts.
Run it · Results · How it works · Limits · GGUF · Data
LvW-Dictate is two open models run one after the other, about 1.5B parameters in total:
| Step | Model | Size | Licence |
|---|---|---|---|
| 1. Speech → transcript | FermionResearch/Phonon-2.5 (25 languages, unchanged) | 0.6B | CC-BY-4.0 |
| 2. Transcript → clean text | this repository: K2-Horizon-0.9B-System1 + a LoRA, fused | 0.9B | Apache-2.0 |
The cleanup step works on English. It removes filler words (um, uh, you know, I mean), repeated words, false starts and self-corrections ("Tuesday, no, sorry, Wednesday" becomes "Wednesday"), and it fixes punctuation and capital letters. It only deletes words: it never adds, rephrases or answers anything. Other languages come out exactly as Phonon-2.5 wrote them.
Real outputs from the held-out test sets:
| Phonon-2.5 transcript | LvW-Dictate |
|---|---|
| Well, I don't want to I don't want this to be a unilateral conversation, but yes, we do. | I don't want this to be a unilateral conversation, but yes, we do. |
| And then and then uh uh the blonde haired girl that used to be on New Heart. | And then the blonde haired girl that used to be on New Heart. |
| I'm not I mean I I like all ranges of movies, but I like that where they run tense. | I like all ranges of movies, but I like that where they run tense. |
| I want to learn coding. Which language should learn, I start with? | I want to learn coding. Which language should I start with? |
| What is the best um, way to form a new habit? | What is the best way to form a new habit? |
Run it
pip install "fermion-research[mlx]" mlx-lm huggingface_hub # Apple silicon
# Linux / Windows: pip install fermion-research torch transformers huggingface_hub
python dictate.py recording.wav --raw
dictate.py is in this repository. It downloads Phonon-2.5 through fermion-research, transcribes the audio, and cleans English transcripts with this model (MLX on Apple silicon, PyTorch elsewhere). --json prints the raw and cleaned text per file. For llama.cpp + transcribe.cpp use the GGUF repository.
To use the cleanup model on its own (for any ASR output), send the transcript as the user turn with the system prompt from dictate.py, greedy decoding:
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("Lucasgitguy-co/LvW-Dictate-1.5B", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Lucasgitguy-co/LvW-Dictate-1.5B", trust_remote_code=True)
Use dictate.py's clean_transcript() rather than the raw model when you can: it adds the English check, splits long text, and runs the deletion check described below.
Results
All test sets are held out: none of their texts or recordings were used for training. WER = word error rate, lower is better.
| Gate | Test set | Phonon-2.5 alone | System1, no training | LvW-Dictate | Pass condition |
|---|---|---|---|---|---|
| G1 cleans up | DisfluencySpeech test (250 real disfluent clips), WER vs the human clean transcript | 16.49 | 12.29 | 7.06 | ≥ 25% lower than Phonon alone |
| G2 does no harm | FLEURS English test (350 sentences), WER vs reference | 6.49 | 6.49 | 6.50 | ≤ +0.3 |
| G2 does no harm | LibriSpeech test-clean (300), WER vs reference | 1.79 | 1.91 | 1.93 | ≤ +0.3 |
| G3 never answers | 194 dictated questions and commands: share returned as the cleaned prompt | 71% | 73% | 99.0% (unseen validation prompts: 96.8%) | ≥ 98%, no output longer than 1.3× its input |
| G4 other languages | 240 FLEURS clips in Phonon's 24 other languages: output identical to Phonon's | 240/240 | 240/240 | 240/240 | all identical |
- Words added by the cleanup step on all four English sets: 0.
- The untrained System1 model, asked to clean up a dictated question, answered it in 125 of 194 cases. Its row above already includes the deletion check, which turned those answers back into the transcript.
- Removing only uh/um by rule (the "Phonon-2.5 alone" column) takes G1 from 16.49 to 14.08; the model brings it to 7.06.
- Unseen validation split (never used for tuning): disfluent speech 17.90 → 7.22 WER (60% lower); 73/73 fluent transcripts returned unchanged; 96.8% of 94 dictated prompts within 10% of the target (G3 bar: 98%; one of the 3 misses is a labelling error). The deletion-check rules were tuned while reading test-set errors, so this is the more honest number for prompts.
- The LibriSpeech increase comes mostly from literary repetition that dictation clean-up removes on purpose ("Nay, nay", "Come, come").
Full numbers: eval/ in this repository.
Training data (Lucasgitguy-co/lvw-dictate-cleanup)
Every input is real Phonon-2.5 output (or a typed prompt), and every target is that input with words removed:
| Part | Source | Input → target | Rows |
|---|---|---|---|
| A | DisfluencySpeech (real disfluent speech) | Phonon's transcript → the same transcript minus the words the human annotation marks as filler, discourse marker or restart | 4,488 |
| B | FLEURS + VoxPopuli (fluent speech) | Phonon's transcript → itself | 4,500 |
| C | the same fluent transcripts | with rule-made fillers, repeats, restarts and self-corrections → the original | 3,970 |
| D | OASST2 first-turn prompts | a dictated question or command (65% with disfluencies) → the cleaned prompt, never an answer | 2,000 |
The targets in part A are built from Phonon's own words, not copied from the human clean transcript. Phonon sometimes mishears (it wrote "peppers" where the speaker said "papyrus"), and the human text would teach the model to write words it never heard. An alignment of Phonon's words to the annotated verbatim text marks which words to drop. Words like "actually", "anyway" and "now" are always kept.
Training
LoRA rank 16 (scale 2.0, dropout 0.05) on all attention and MLP projections of all 28 layers, learning rate 1e-4 with cosine decay, batch 8, 2 epochs (3,740 steps), loss on the answer only, MLX on a 16 GB MacBook Air M4. The run stopped once on GPU memory at step 2,090 (another job was using the GPU) and was resumed from the epoch-1 checkpoint with the schedule continued. Epoch 2 was chosen over epoch 1 because, at equal G1, it deletes far fewer real words on fluent speech.
The deletion check
After the model answers, its output is aligned to the transcript. Each run of deleted words must be one of:
- fillers ("uh", "you know", "I mean"; "well", "so", "like" only when followed by a comma or opening a sentence);
- a repeat or restart of the words next to it;
- a self-correction ("…, no, sorry, …").
Anything else is put back. If the output contains a word the transcript doesn't, the transcript is used as it is. Finally, an output shorter than 40% or longer than 110% of the transcript falls back to the transcript. This makes the cleanup deletion-only by construction. On the test sets the check put back words the model had dropped in 52 of 250 DisfluencySpeech clips, 1 of 350 FLEURS, 10 of 300 LibriSpeech and 1 of 194 prompts.
English check
A word-list test (is_english in dictate.py, standard library only) decides whether a transcript is cleaned. On held-out clips it called 0 of 240 non-English transcripts English, and missed 12 of 900 English ones (1.3%; those are returned uncleaned).
Limits
- English only for the cleanup. K2-Horizon was trained on English and Chinese, so Dutch, German and the rest are passed through untouched.
- It cannot fix words Phonon-2.5 misheard; it only removes words.
- It removes intentional repetition too ("very, very good" can become "very good").
- A self-correction split by a full stop ("…on Tuesday. No, sorry, Wednesday…") is often left in.
- Trained on short utterances (up to ~30 s of speech); longer text is split into ~120-word pieces at sentence ends.
- Phonon-2.5 is used as published; its accuracy per language is on its card.
Licence and credits
The weights and code here are Apache-2.0. The model comes from IFM/K2-Horizon-0.9B (Apache-2.0) via K2-Horizon-0.9B-System1. Speech recognition: Phonon-2.5 by Fermion Research (CC-BY-4.0, derived from NVIDIA parakeet-tdt-0.6b-v3), not included in this repository. Training text: DisfluencySpeech (Apache-2.0; utterances from Switchboard-1), FLEURS (CC-BY-4.0), VoxPopuli (CC0), OASST2 (Apache-2.0). See NOTICE.