Fleet-Clean-0.36B
Speech in, clean text out. It is two open models with 363M parameters in total:
- Fleet-2 (56M, Cactus Whistle) writes down what was said.
- A word tagger (307M) cleans that transcript. It is the encoder of convaiinnovations/laya-multilingual (mmBERT) with a new head that decides, for every word:
- keep or delete it (fillers, discourse markers, repeats, false starts)
- which mark follows it (none , . ? !)
- whether its first letter changes case
The tagger only removes Fleet-2's words and fixes punctuation and capitals. It never writes a word of its own, so it can't invent content or answer what was dictated.
GGUF version: Fleet-Clean-0.36B-GGUF · Training data: fleet-clean-tags · Collection: Speech recognition
Usage
pip install cactus-compute torch "transformers>=5" safetensors
hf download Lucasgitguy-co/Fleet-Clean-0.36B --local-dir fleet-clean && cd fleet-clean
python fleet_clean.py clip.wav # clean transcript
python fleet_clean.py --json clip.wav # raw, clean and the per-word tags
python fleet_clean.py --text "so I I think, you know, we should go"
python fleet_clean.py --punct full clip.wav # also re-punctuate words away from deletions
Audio is 16 kHz WAV (ffmpeg -i in.m4a -ar 16000 -ac 1 clip.wav). Fleet-2 detects the language, or you can pass --language. The delete threshold is 0.5. It was calibrated on validation data so that fluent speech keeps its words, and --threshold changes it.
Examples
These are from the DisfluencySpeech test set (English):
- Fleet-2: Yeah. Well, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly. Fleet-Clean: Yeah, robots are darling. That would be fun if you could get them trained. Otherwise, they're pretty smelly.
- Fleet-2: Well, McNew, New, and Lara, I guess it's leaving one of them is a leaving. Fleet-Clean: McNew, New, and Lara, I guess one of them is a leaving.
- Fleet-2: Well, I don't want to. I don't want this to be a unilateral conversation, but yes, we do. Fleet-Clean: I don't want to. I don't want this to be a unilateral conversation, but yes, we do.
Evaluation
Real disfluent English speech. Fleet-2 transcribes the audio, and the output is compared with DisfluencySpeech's own cleaned transcript:
| DisfluencySpeech test (250 clips) | Fleet-2 alone | Fleet-Clean |
|---|---|---|
WER against the cleaned transcript (transcript_c), punctuation ignored |
18.8% | 14.6% |
Punctuation F1 against transcript_c |
0.6700 | 0.6695 |
Fluent speech (FLEURS test). These are the same 100 clips per language as Fleet-2's card. A cleanup model should leave fluent speech alone:
| Language | Words deleted from fluent speech | WER before | WER after |
|---|---|---|---|
| English | 0.47% | 14.5% | 14.3% |
| German | 0.05% | 22.0% | 22.0% |
| French | 0.04% | 24.9% | 25.0% |
| Spanish | 0.05% | 11.9% | 11.9% |
| Italian | 0.06% | 16.7% | 16.7% |
| Dutch | 0.19% | 30.1% | 30.0% |
| Polish | 0.29% | 41.6% | 41.6% |
| Korean | 0.27% | 42.8% (CER) | 42.9% (CER) |
Fillers, discourse markers, repeats, restarts and self-corrections were inserted into Fleet-2 transcripts of held-out VoxPopuli / Zeroth / FLEURS clips. The table shows whether the tagger finds exactly the inserted words. "Weak" means F1 below 0.80.
| Language | Delete F1 | Precision | Recall |
|---|---|---|---|
| English | 0.981 | 0.967 | 0.996 |
| German | 0.984 | 0.968 | 1.000 |
| French | 0.970 | 0.946 | 0.995 |
| Spanish | 0.971 | 0.978 | 0.964 |
| Italian | 0.969 | 0.959 | 0.979 |
| Dutch | 0.952 | 0.920 | 0.986 |
| Polish | 0.990 | 0.980 | 1.000 |
| Korean | 0.979 | 0.975 | 0.984 |
Limits:
- Real disfluent speech was only tested in English. The other seven languages were trained and tested on synthetic disfluencies, so their numbers show how well the tagger removes those inserted patterns, not how it handles real conversation.
- Fleet-2 already leaves out most "uh/um". It learned from Whisper transcripts. What is left to clean is mostly "you know", "I mean", "like", repeats and restarts.
- Recognition errors stay. The tagger can only delete or re-punctuate words, so it can't fix a misheard word.
- Clips are up to 30 s per call. The tagger reads up to 512 tokens and leaves anything past that unchanged.
How it was built
Data, training and gates- Transcripts: Fleet-2 (
whistle.cact, native Cactus engine, automatic language detection) transcribed:- DisfluencySpeech train/validation/test
- VoxPopuli validation (7 languages, a split Fleet-2 never trained on)
- Zeroth-Korean train minus Fleet-2's own training rows
- FLEURS Korean train minus Fleet-2's rows
- Labels:
- English deletions: each Fleet-2 word is aligned to DisfluencySpeech's verbatim transcript, and that transcript's words are mapped to its cleaned version.
- Punctuation and case: taken from the human reference wherever Fleet-2's word matches it. Zeroth has no punctuation, so its rows only teach deletion.
- Synthetic rows: disfluencies are only ever inserted, so the clean text is always recoverable by deletion.
- Training:
- Laya-multilingual's encoder, with its 256k-token embedding table frozen, plus a new 768→10 head.
- PyTorch on an Apple M4 (MPS), fp32.
- Best validation checkpoint kept, then the delete threshold calibrated on validation only.
- Release gates:
- G1: cleaning lowers WER on real disfluent English, and punctuation F1 doesn't drop. WER dropped from 18.76% to 14.61%, but punctuation F1 went from 0.6700 to 0.6695 (−0.0005), so this part of the gate did not pass. It was published anyway by the owner's decision, and the number is reported as measured.
- G2: at most 1.0% of words deleted from fluent FLEURS speech in every language. Passed in all 8.
- Punctuation mode: by default, Fleet-2's own punctuation is kept, and the tagger's punctuation and case are used only next to removed words. This was chosen on DisfluencySpeech validation, where re-punctuating everything scored lower against the human clean transcripts.
--punct fullre-punctuates every word; on fluent validation speech in all 8 languages, full mode was slightly more accurate than Fleet-2's own marks.
License and attribution
Apache-2.0. See NOTICE for every component:
- Fleet-2 / Cactus Whistle (Apache-2.0)
- Laya-multilingual (Apache-2.0), which builds on mmBERT (MIT)
- DisfluencySpeech (Apache-2.0)
- VoxPopuli (CC0)
- Zeroth-Korean and FLEURS (CC BY 4.0)
No audio is included.