wav2vec2-base-sk-17k-fav-kky-aligner

A Slovak Wav2Vec2ForCTC model, fine-tuned to be used as a forced aligner — not as a general-purpose transcriber. Given audio and its known transcript, it produces accurate word/character-level timestamps. It can also free-decode (greedy CTC), and that's how it's evaluated below, but that's not the job it was built for.

Where it comes from

The encoder is fav-kky's Slovak wav2vec2-base, pretrained self-supervised on ~17,000 hours of Slovak speech. This repo adds a fresh 46-token CTC head and fine-tunes the whole model on labeled Slovak speech.

The vocabulary is fixed and small on purpose: lowercase letters + Slovak diacritics only (a-z áäčďéíĺľňóôŕšťúýž), no punctuation, no digits. A CTC head can't hear a comma or a capital letter, and for an alignment model, extra output classes just blur the frame posteriors you're trying to align against.

Training data

Fine-tuned on the official train splits of five Slovak corpora, capped at 15 minutes of audio per speaker to keep a few over-represented voices from dominating:

Utterances containing digits were dropped rather than expanded (2019 is spoken dvetisícdevätnásť, and Slovak numerals decline by case, so naive expansion produces grammatically wrong training labels). About 43 hours were dropped this way, affecting only 19 speakers.

One side effect worth knowing: SloPalSpeech and VoxPopuli make up ~86% of the training hours, and both are long-form audio (20–30 second clips). The model has a strong prior toward long, connected speech and is noticeably less comfortable with isolated short clips (see Limitations).

Results

Word/character error rate, full test split of each corpus, greedy CTC decoding (no language model):

¹ Common Voice is dominated by very short clips (one or two words), which this model over-emits on due to its long-form training bias — WER on 7+ word utterances (half the test set) is 13.7%, close to the other corpora. CER, which is less sensitive to a handful of runaway short clips, tells the more honest story here.

Since the model trained on these corpora's own train splits, these numbers aren't fully independent recognition — they're best read as a health check on the frame posteriors, not a leaderboard score.

Intended use

This model is meant to be used the way forced aligners are used: given audio and its transcript, align text to audio frames (e.g. with torchaudio.functional.forced_align) to get word or character timestamps. In that setting the model never has to guess what was said, only when — so the short-clip over-emission problem above doesn't come into play.

Free-decoding this model as a standalone transcriber will work reasonably on longer, connected speech, but will be unreliable on short utterances and has no punctuation or capitalization in its output.

Limitations

• No punctuation, no capitalization, no digits in the vocabulary.
• Weak on very short (1–3 word) isolated utterances due to the long-form training data.
• Not evaluated as an aligner directly (i.e. against ground-truth timestamps) — the numbers above are free-decode WER/CER, used as a proxy for posterior quality.