stt_cy_parakeet_tdt_ctc_110m

Welsh (Cymraeg) acoustic model for Home Assistant Speech-to-Phrase: on-device recognition of Welsh voice commands. Made by Uned Technolegau Iaith (techiaith), Bangor University.

Fine-tuned from nvidia/parakeet-tdt_ctc-110m. The CTC branch is exported to ONNX with the same k2-fsa/sherpa-onnx script that produced the Open Home Foundation's English default, so the graph and interface match the English model.

This is the far-field round-1 model, trained with real room reverberation. The first model is kept under the tag run1.

What is here

Using it in Speech-to-Phrase

Speech-to-Phrase needs three things for Welsh:

• the Welsh speech_to_phrase sentences in home-assistant-intents (proposed to OHF-Voice/intents);
• the model registered in OHF-Voice/apps (speech-to-phrase/src/models.py): "cy": {"nemo": "stt_cy_parakeet_tdt_ctc_110m"} and MODEL_MAX_SCORE["stt_cy_parakeet_tdt_ctc_110m"] = 4.0;
• the package in the model store the add-on downloads from.

The score gate of 4.0 is required. The generic NeMo default of 5.0 would accept everyday speech: on real recordings, correctly accepted commands score at most 3.94 and everyday sentences at least 4.67.

Text is lowercase and unpunctuated, numbers are words and hyphens are spaces, which is what Speech-to-Phrase's normaliser produces. results/cynorm.py is the normaliser used for training.

Licence

Released under the Apache License 2.0. The accompanying NOTICE carries the attributions required by the base model's and the training data's licences, and the warnings below. Section 4 of the licence requires anyone who redistributes the model or a derivative to keep that NOTICE.

The model is a modified version of nvidia/parakeet-tdt_ctc-110m, © NVIDIA, licensed under CC BY 4.0; those terms continue to apply to the base model's part of it. It is not affiliated with, endorsed by or sponsored by NVIDIA.

One-line attribution: Includes stt_cy_parakeet_tdt_ctc_110m, developed by the Language Technologies Unit, Bangor University, and released under the Apache License 2.0.

Intended use and limitations

The model recognises spoken Welsh commands on device, with Home Assistant's Speech-to-Phrase: it picks the command, from a fixed list of sentences, that best matches what was said. It is a research model, not a certified product, and is not a substitute for a qualified transcriber.

Automatic speech recognition makes mistakes. Output can be wrong while appearing confident. Known failure modes include recognising a command when someone said something else that sounds similar (for example a question the list does not cover); dropping a word, such as part of a device name; taking an English device or room name for a Welsh one; and reduced accuracy with distant microphones, background noise, and accents or dialects outside the training data. The model is used interactively and in real time: tell users that what is recognised is automatic and may be wrong. Have any transcript kept as a record checked and corrected by a competent person before it is relied on, used in any decision affecting a person, or shared or published.

Not fit for medical, legal, financial, safety-critical or emergency settings, or any decision affecting a person's rights, health, liberty or livelihood, unless a qualified person reviews every output.

Use in Wales. The model was developed to help organisations provide services in Welsh that are as good as those they provide in English, not to replace Welsh-language provision. Under section 1 of the Welsh Language (Wales) Measure 2011, Welsh has official status in Wales, given legal effect by, among other things, the treatment of Welsh no less favourably than English and by standards of conduct made and enforced under Part 4 of the Measure. Bodies still operating a Welsh language scheme under Part II of the Welsh Language Act 1993 must give effect to the principle in section 5 of that Act that Welsh and English are treated on a basis of equality. Which duties apply depends on the body and the regulations or scheme that cover it, and responsibility for compliance rests with the body deploying the model. See the Welsh Language Commissioner's regulatory policy statement on artificial intelligence and the Welsh language, Technology, Websites and Software: Welsh Language Considerations, and the advice document Bilingual drafting, translation and using Welsh face to face. The legislation: Welsh Language (Wales) Measure 2011 and Welsh Language Act 1993.

How good is it

Real Welsh speech. A held-out test set (never used for training or model choice): 82 prompts read by four speakers from north and south Wales, on close microphones and laptop microphones at arm's length, 328 takes. Speech-to-Phrase with the Welsh grammar, gate 4.0:

The two arm's-length speakers gain the most margin: the three lowest everyday sentences scored 4.56, 4.90 and 5.13 with run1, and 4.67, 5.91 and 6.49 with this model.

On a Home Assistant Green (RK3566, four Cortex-A55 cores, 4 GB), fp32, with the same architecture as run1: a median of 2.0 s from the end of speech to the transcript, about 0.55 s per second of audio, and 0.8 GB of memory. Commands spoken quietly from across the room through a Home Assistant Voice PE were recognised.

Far-field dev set. 600 commands spoken by the seven cy_GB Piper voices and 519 Common Voice Welsh sentences, each in four versions with reverberation from 13 rooms never used in training. Commands correct of 600 and false accepts of 519:

Open-ended recognition is not what this model is for: it has no language model, and in Speech-to-Phrase the grammar is the language model. For reference, run1 on techiaith/asr-benchmark-cy-en-2606 (WER %, case-normalised / fully normalised):

Training data

The training data is not distributed with the model and is not licensed under the Apache License; each source keeps its own terms. 294.69 h of speech (Welsh 262.2 h, English 32.5 h). Audio was deduplicated by content hash against every test set in the benchmark, and talentau-llais was capped at 3,400 utterances per speaker.

This model then had 5,000 more updates on the same data, with a 0.5 chance per utterance of being convolved with a measured room impulse response: 1,653 RIRs from 37 rooms, balanced to 30 per room, each modified (one channel, resampled to 16 kHz, trimmed, truncated at the noise floor, faded and peak-normalised). The checkpoint at step 4,600 was chosen on the far-field dev set.

MODEL_CARD.md has the training settings.

Known issues

• Short questions that start like a command can be accepted as that command: "beth yw'r penawdau" as "beth yw'r tywydd". This needs a grammar or scoring fix, not training.
• On the Piper dev set, about 22 of 142 compound numbers lose their first part ("deuddeg" → "deg"). In real speech every compound number in the test set was recognised ("bymtheg", "dau ddeg", "un deg pum", "dri deg"); "deuddeg" and "deunaw" are not yet tested.
• Request forms such as "wnewch chi …" score close to the gate, because the on-device grammar has no skip words.