Qwen3-ASR-0.6B-Jev-Dutch

A decision model that describes Dutch speech clips: who is speaking (gender, age, child or adult, Netherlands or Flemish Dutch) and what the recording is like (speech or not, how many speakers, overlap, noise, reverberation, music, telephone, overall quality). It is built for filtering and enriching Dutch speech data: a single model with one pass over the audio with 12 calibrated answers per clip, at about 35 ms per clip on one A100.

It is Qwen/Qwen3-ASR-0.6B finetuned with decision heads, following Qwen-Audio-Jev (arXiv 2610.21614).

What it can be used for

• Cleaning ASR training data: drop clips without speech, with several or overlapping speakers, with music, strong noise or reverberation, or that are not Dutch, before training or finetuning a speech recogniser.
• Selecting TTS and voice data: keep single-speaker, close-microphone, "good"/"excellent" quality clips, per gender, age group and NL/BE variety.
• Describing and balancing a corpus: how many women, children, seniors and Flemish speakers a dataset contains, for a datasheet, for rebalancing, or to spot gaps before training.
• Adding metadata to unlabelled audio (broadcast archives, podcasts, interviews, lectures): per speaker turn together with a diarization (pyannote), or per time window for events such as music, noise and overlap.
• Quick searches in large collections: find children's speech, telephone recordings or music fragments.

How it works: decision heads

The audio goes once through the speech encoder. The decoder then reads it together with a short Dutch question (for example "Is de spreker een man of een vrouw?"), and the hidden state at the end of the question goes into one of three small heads, instead of generating text:

• choice head: picks one of a few options (gender, NL/BE);
• score head: picks a level on an ordered scale (age decade, noise, reverberation, quality) and also gives the expected level;
• yes/no head: one probability (speech, more than one speaker, overlap, music, telephone, Dutch).

With signal=True, predict also returns plain signal statistics (duration, level, clipping, silence, rough SNR, bandwidth) that do not come from the model.

Training data:

Recording conditions are labelled by mixing them in during training (second talkers, simulated rooms from OpenSLR 28 and 2,000 of our own, MUSAN noise and music, narrowband telephone, no-speech clips); quality labels come from DNSMOS P.835.

What it does not cover (yet)

The model answers only the 12 questions above. It does not:

• transcribe speech or judge transcripts (word error rate, transcript quality); use Qwen3-ASR itself or another speech recogniser for that;
• give other speaker attributes, such as education level, regional dialect, native language, emotion or speaking style;
• give other measurements, such as speech rate, pauses or exact SNR and reverberation time in dB or seconds: noise, reverberation and quality are levels. Only the optional signal=True statistics are plain measurements.

The answers are estimates for analysing groups of clips. Do not use them for decisions about individual people, or to identify, verify or profile speakers. See Limitations at the end for where the current answers are weaker.

Usage (Transformers)

pip install "transformers>=5.20" torch soundfile librosa

from transformers import AutoModel

jev = AutoModel.from_pretrained("tvosch/Qwen3-ASR-0.6B-Jev-Dutch", trust_remote_code=True).to("cuda")  # or CPU

jev.predict("clip.wav")                       # also a list of paths, encoded bytes, or arrays with sr=...
# [{"gender": {"label": "female", "confidence": 0.99, "probs": {...}},
#   "age": {"label": "40s", ...}, "age_group": {"label": "40-59", ...}, "child": {"p": 0.01},
#   "variety": {"label": "NL", ...}, "noise_level": {"label": "none", ...}, "quality": {"label": "good", ...},
#   "channel_music": {"p": 0.02}, "multi_speaker": {"p": 0.01}, "has_speech": {"p": 1.0}, ...}]

jev.predict(["a.wav", "b.wav"], heads=("gender", "age"))           # only some answers (faster)
jev.predict("long.wav", window=5)                                   # long recordings: one row per 5 s window
rows = jev.predict("long.wav", segments=[(0.0, 4.2, "A"), (4.2, 9.0, "B")])   # e.g. turns from pyannote
jev.by_speaker(rows)                                                # one answer per speaker

A recording longer than 20 s gets one averaged answer by default; with several speakers that answer describes no one, so use window= for events and segments= from a diarization for speaker answers.

Comparison with specialist tools

This Transformers export gives the same label for 98.8–100% of clips per answer (2,358 clips of real broadcast and spoken audio).

Performance

Recording conditions on identical audio (1,700 clips with held-out rooms, noise and music; AUC). The labels come from this model's own mixing procedure, which favours it:

Dutch speech of children and non-native speakers (, never trained on), share of clips called "not Dutch": Qwen3-ASR-0.6B-Jev-Dutch 1–28% by group, VoxLingua107 13–67%.

Speaker answers on (500+ speakers, held out; gender as balanced accuracy per speaker, age group as share of clips right, Netherlands–Flanders range):

Child vs adult: Qwen3-ASR-0.6B-Jev-Dutch 0.81, audeering 0.54 (balanced accuracy); child (≤ 12) AUC 0.94. audeering = wav2vec2-large-robust-24-ft-age-gender.

Other results:

• Speakers, per speaker: Common Voice nl (137 speakers) gender 0.99, age QWK 0.84, NL/BE 0.95; VoxPopuli nl (25–31 speakers) gender 1.00, age QWK 0.66, NL/BE 1.00.
• Recording conditions on material never seen in training (real rooms, recorded noise, other music): AUC 0.995–1.00 for telephone, no speech, music, overlap and more than one speaker; noise 0.98; real-room reverberation 0.93. Levels: noise accuracy 0.84, reverberation 0.90. On : AUC 0.99–1.00 for overlap, music, more than one speaker and no speech.
• Quality ranks clips like DNSMOS (Spearman 0.57–0.64 on Dutch sets); it is a ranking, not a MOS score.
• A real Dutch broadcast (60 clips labelled by a person): gender 59/60, speech 60/60, music AUC 0.97, more than one speaker AUC 1.00 (only 4 such clips).

Ablations

What we tried on the way (Dutch results first; each is one training run unless noted, so differences under 0.02 are within seed noise for Dutch adults):

• Other languages in training. Every model mixes 50% Dutch with 50% other languages, because the Dutch data has few women (25%), children (24 speakers under 20) and seniors (9 over 70). Adding more other-language age data with age rebalancing improved age in those languages (speechocean762 children QWK 0.08 → 0.63, Spanish 0.66 → 0.78) but not Dutch age, and cost children's gender. The child answer itself comes from speechocean762 (Mandarin-speaking children) and transfers to Dutch children (AUC 0.94). A Dutch-only model was not trained, so the gain from mixing for Dutch adults is not measured.
• Gender balancing among children only flips the default guess (girls 95% / boys 11%); children's voices carry little gender signal, hence "unknown".
• Flemish ×2 weight raised NL/BE on for Flemish teenagers (0.72 → 0.93) and seniors (0.68 → 0.76), read speech; the same run also added transcript features, so the two are not separated.
• A transcript-quality head was dropped: the ASR model's own token probabilities are better (AUC 0.88–0.93) and free.
• A separate Dutch age temperature (no retraining) halved the Dutch age calibration error (0.21 → 0.11 on Common Voice nl, 0.23 → 0.14 on VoxPopuli).

Limitations

• "Sounds like Dutch" reacts to accent (28% of non-native adults' clips called not Dutch); do not use it to exclude speakers.
• Teenagers are mostly placed in 20-39 and seniors in their 60s; gender of children under 12 is not reliable.
• NL vs BE means "Netherlands Dutch or not": a foreign accent pushes it towards BE.
• Real-room reverberation is missed about a third of the time at 0.5; real telephone audio is untested.
• Dutch training speakers are mostly adult men; VoxPopuli nl test speakers overlap with the official VoxPopuli test split, so do not benchmark on it.

Precision and speed

16-bit gives the same answers as float32 (99–100% of labels the same on 2,358 clips) at about twice the speed. Use bf16 on recent GPUs and on CPUs with bf16 instructions, float16 on older GPUs, and float32 on other CPUs:

import torch
from transformers import AutoModel

repo = "tvosch/Qwen3-ASR-0.6B-Jev-Dutch"
jev = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.bfloat16).to("cuda")    # A100, H100, RTX 30xx+
jev = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.float16).to("cuda")     # T4, V100, RTX 20xx
jev = AutoModel.from_pretrained(repo, trust_remote_code=True, dtype=torch.float32)                # CPU without bf16
jev.predict(["clip1.wav", "clip2.wav"], batch_size=16)

The decision heads always compute in float32. int8 (jev.quantize_int8()) changes the answers too much and is not recommended.

Speed in milliseconds per clip (200 clips of about 6 s; specialist tools in float32):

*pyannote segmentation-3.0, PANNs CNN14, brouhaha, a bandwidth check and VoxLingua107. CPU: AMD EPYC 9654 (Zen 4, with bf16 instructions). On CPUs without bf16 instructions, float32 takes about 1.5 s per clip, and the specialist tools are faster.

Citation and attribution

Base model: Qwen3-ASR (Qwen team). Method: Qwen-Audio-Jev (arXiv 2610.21614). CC-BY data and models: Multilingual LibriSpeech (Pratap et al., 2020), MUSAN (Snyder et al., 2015), DNSMOS (Reddy et al., 2021, 2022; quality labels).