Bolo Sun v1: Hinglish voice-command speech recognition

Bolo Sun is the speech recogniser in Bolo, an offline Hinglish voice-command assistant for laptops. It takes short spoken commands such as "downloads folder mein terminal kholo" or "volume 40 pe kar do" and writes them in Roman-script Hinglish.

Base: Oriserve/Whisper-Hindi2Hinglish-Swift (whisper-base size, 73M parameters).
Training data: fully fine-tuned on synthetic speech only. That means 3,287 command clips covering 20 command types, made with Indic Parler-TTS, with real room impulse responses and noise (OpenSLR 28) and about 3% noise-only clips. Training ran for 5 epochs, about 12 minutes on a free Kaggle T4.
Format: float16, 142 MB. On an Apple M2 it takes about 100 ms per command with greedy decoding and about 180 ms with beam 5.

Use

from transformers import pipeline

asr = pipeline("automatic-speech-recognition", model="kungfupanda12/bolo-sun")
print(asr("command.wav", generate_kwargs={"language": "en", "task": "transcribe"})["text"])

Passing a file path needs ffmpeg installed; otherwise pass {"raw": samples, "sampling_rate": 16000}.

The full assistant does more than transcribe:

it keeps the 5 best transcripts;
it reranks them with the Bolo Tiny command model;
it checks names against the apps and folders that really exist on the machine.

See the GitHub repo.

Results

Test set	Swift (base)	Bolo Sun
Synthetic, held-out TTS voices: WER	32.1%	16.2%
Real speech, 55 commands, 1 speaker: WER	–	18.3% (v0: 23.7%)
Real speech, same 55: correct action (full Bolo stack)	about 51%	80.0%

Limits on the real-speech numbers:

They come from one speaker (the author), using a laptop mic in a quiet room. With n = 55 the uncertainty is about ±11 points.
Some pipeline fixes were found on this same set, so 80% is optimistic.
Treat this as a research preview until it is tested on more voices.

Known weaknesses:

English words inside Hindi sentences ("quit arc" heard as "width arduino", "google" as "go well").
Slight forgetting on long general speech. On the OpenSLR 104 Hindi-English test it was v0: 59.1% WER vs Swift's 55.2%.
Trained on commands, not dictation.

Every experiment, including the failures, is in WHAT_WE_TRIED.md. One example: training for 10 epochs lowered synthetic WER but made real speech worse.

Data and privacy

No recordings of real people saying commands were used in training.
The author's own test clips stay on the author's laptop.
Bolo runs fully offline, so no audio leaves the device.

License

Apache-2.0, the same as the base model and the TTS teacher.

加载中…

Bolo Sun v1: Hinglish voice-command speech recognition

Base: Oriserve/Whisper-Hindi2Hinglish-Swift (whisper-base size, 73M parameters).

Training data: fully fine-tuned on synthetic speech only. That means 3,287 command clips covering 20 command types, made with Indic Parler-TTS, with real room impulse responses and noise (OpenSLR 28) and about 3% noise-only clips. Training ran for 5 epochs, about 12 minutes on a free Kaggle T4.

Format: float16, 142 MB. On an Apple M2 it takes about 100 ms per command with greedy decoding and about 180 ms with beam 5.

Use

from transformers import pipeline asr = pipeline("automatic-speech-recognition", model="kungfupanda12/bolo-sun") print(asr("command.wav", generate_kwargs={"language": "en", "task": "transcribe"})["text"])

Passing a file path needs ffmpeg installed; otherwise pass {"raw": samples, "sampling_rate": 16000}.

The full assistant does more than transcribe:

it keeps the 5 best transcripts;

it reranks them with the Bolo Tiny command model;

it checks names against the apps and folders that really exist on the machine.

Results

Test set	Swift (base)	Bolo Sun
Synthetic, held-out TTS voices: WER	32.1%	16.2%
Real speech, 55 commands, 1 speaker: WER	–	18.3% (v0: 23.7%)
Real speech, same 55: correct action (full Bolo stack)	about 51%	80.0%

Limits on the real-speech numbers:

They come from one speaker (the author), using a laptop mic in a quiet room. With n = 55 the uncertainty is about ±11 points.

Some pipeline fixes were found on this same set, so 80% is optimistic.

Treat this as a research preview until it is tested on more voices.

Known weaknesses:

English words inside Hindi sentences ("quit arc" heard as "width arduino", "google" as "go well").

Slight forgetting on long general speech. On the OpenSLR 104 Hindi-English test it was v0: 59.1% WER vs Swift's 55.2%.

Trained on commands, not dictation.

Every experiment, including the failures, is in WHAT_WE_TRIED.md. One example: training for 10 epochs lowered synthetic WER but made real speech worse.