NB-ASR Bokfink02 — beta

Private beta model from the NB-ASR project at the National Library of Norway. This is a Norwegian-focused adaptation of the Qwen3-ASR-0.6B family, with reading-optimised and verbatim output, Bokmål/Nynorsk written targets, and Norwegian-speech-to-English translation. It also learns from long recordings, sparse-speech recordings, contextual vocabulary and intentionally empty targets.

This is an experimental checkpoint for authorised evaluation and integration, not a production guarantee. Model weights do not include access rights to the underlying speech datasets. The repository is private, not public-and-gated.

What to use it for

Nynorsk and English training targets generally reuse Norwegian source audio; these are output-language controls, not evidence of an independently trained English-speech corpus. The original model's other languages have not been systematically revalidated after this adaptation.

Verbatim aims to retain spoken repetitions, repairs and hesitations. It does not guarantee exact reproduction of every disfluency. Reading-optimised output aims at readable written text; it is not summarisation. There is no hard-coded rule to cap repetitions at three. Neither <summary> nor the reserved <reading_optimised> token was used as a training style in this run.

Installation and private access

Use a fresh Python 3.12 environment with a suitable CUDA-enabled GPU. The release examples target qwen-asr==0.0.6 / Transformers 4.57.6. That package's vLLM extra pins vLLM 0.14.0; use a compatible CUDA/PyTorch environment rather than upgrading individual components independently.

python -m pip install "qwen-asr==0.0.6" huggingface_hub soundfile scipy
hf auth login
hf download NbAiLab/nb-asr-bokfink02-beta --local-dir ./nb-asr-bokfink02-beta
cd nb-asr-bokfink02-beta

Your HF account/token must have read access to this private repository. Do not put tokens in source files, notebooks, shared logs or command-line examples. For reproducible deployment, download a fixed Hub commit with --revision; the model's release provenance is included in release_provenance.json.

The accompanying inference.py is a small, optional local helper. Read it before running it. It does not install custom model implementations or require trust_remote_code=True; Qwen classes come from the installed qwen-asr package.

Prompt contract: three independent controls

Always load the tokenizer, processor and chat_template.jinja from this repository. The custom tokenizer remaps reserved slots without changing the embedding dimensions. Do not substitute the original Qwen tokenizer or one from another NB-ASR release.

The conceptual serialisation for keyword-conditioned verbatim Bokmål is:

<|im_start|>system
metoprolol Tromsø Nordlandssykehuset<|im_end|>
<|im_start|>user
<|audio_start|><|audio_pad|><|audio_end|>
<verbatim><|im_end|>
<|im_start|>assistant
language Norwegian<asr_text>

Use the processor to create this prefix; do not hand-tokenise the audio markers. The generated continuation is the transcript only. Leave SYSTEM empty when no keywords are needed. Do not place <verbatim> in the system message or put nob, nno, eng into the user message as instructions. Do not substitute Norwegian Bokmål for the trained Norwegian header.

The upstream Qwen3ASRModel.transcribe(language=...) validates against its original language list, which does not contain our Norwegian selectors. Also, its context parameter supplies SYSTEM text: context="<verbatim>" is wrong for this checkpoint. The helper uses explicit processor prompts and the ordinary Transformers/vLLM generation backends, avoiding both pitfalls without globally patching the upstream library.

Optional keyword boosting

Supply a short list of plausible names/terms, joined by spaces, preserving multiword phrases. No Vocabulary: prefix, comma-separated list, true/distractor labels or instructions are needed. For example:

metoprolol Eliquis Tromsø Nordlandssykehuset

Keywords are hints, not text that must be inserted. Most training rows had no keywords: 96.05% empty SYSTEM context; 2.24% true-plus-distractor context; 1.12% distractors only; 0.59% true keywords only. The contextual feature has not been separately benchmarked for this beta release. Prefer ordinary no-context ASR unless useful vocabulary is actually available.

Transformers: quickstart

The following commands use the downloaded helper and local checkpoint:

# Reading-optimised Bokmål
python inference.py /path/to/audio.wav --model . --target-lang nob

# Verbatim Bokmål
python inference.py /path/to/audio.wav --model . --target-lang nob --style verbatim

# Written Nynorsk
python inference.py /path/to/audio.wav --model . --target-lang nno --style verbatim

# English translation of Norwegian speech
python inference.py /path/to/audio.wav --model . --target-lang eng

# Verbatim with optional keyword context
python inference.py /path/to/audio.wav --model . --style verbatim \
  --keywords "metoprolol Eliquis Tromsø Nordlandssykehuset"

Output is JSON containing text, target_lang, style, duration_seconds, truncated and optional timestamps. The helper downmixes to mono and resamples to 16 kHz without normalising away quiet voices or applying VAD. It uses BF16, greedy decoding and explicitly enables the generation KV cache; this overrides the checkpoint's training-time use_cache=false setting without changing weights.

For Python integration, run from the downloaded directory:

from inference import BokfinkASR

model = BokfinkASR(model=".", backend="transformers", device="cuda:0")
result = model.transcribe(
    "/path/to/audio.wav",
    target_lang="nob",
    style="verbatim",
    keywords="",  # SYSTEM vocabulary; not the style tag
)
print(result["text"])

vLLM: transcription

Install the supported vLLM extra in a suitable environment:

python -m pip install "qwen-asr[vllm]==0.0.6"
CUDA_VISIBLE_DEVICES=0 python inference.py /path/to/audio.wav \
  --model . --backend vllm --target-lang nob --style verbatim \
  --gpu-memory-utilization 0.60 --max-model-len 32768 --max-new-tokens 4096

For reading-optimised output, omit --style verbatim. Nynorsk, English and keywords use exactly the same switches as the Transformers example.

Programmatic vLLM entry points must use the multiprocessing main guard:

from inference import BokfinkASR

if __name__ == "__main__":
    model = BokfinkASR(
        model=".", backend="vllm", gpu_memory_utilization=0.60,
        max_model_len=32768, max_new_tokens=4096,
    )
    print(model.transcribe("/path/to/audio.wav", target_lang="nob", style="verbatim"))

This is offline vLLM inference with the same explicit prompt contract, not an OpenAI-compatible HTTP server. The helper constructs requests with rendered prompt and multi_modal_data={"audio": [waveform]}. For a separate serving integration, preserve all three controls above: generic transcription APIs or chat templates that omit the assistant-language prefix can change the task.

The example limits concurrent sequences to one to give a conservative starting point for long audio. Tune batch/concurrency and memory utilisation on your own hardware; no universal minimum VRAM or throughput claim is made here.

vLLM or Transformers together with the forced aligner

Use the Norwegian-adapted companion NbAiLab/nb-asr-beta-qwen06b-08052026-forced-aligner. The helper pins its revision to 6ddd1268709255a14f576b4db898942eb24b8e38. It is a separate model, not part of the ASR weights. The aligner runs through Transformers even when ASR runs through vLLM; reserve GPU memory for both, or place them on separate devices. Separate HF access may be required for it.

For a short recording:

CUDA_VISIBLE_DEVICES=0 python inference.py /path/to/audio.wav \
  --model . --backend vllm --target-lang nob --style verbatim \
  --gpu-memory-utilization 0.60 --timestamps

Or in Python:

from inference import BokfinkASR

if __name__ == "__main__":
    model = BokfinkASR(model=".", backend="vllm", gpu_memory_utilization=0.60)
    model.load_aligner(device="cuda:0")
    result = model.transcribe("/path/to/audio.wav", style="verbatim", timestamps=True)
    print(result["text"])
    for word in result["timestamps"]:
        print(word["text"], word["start"], word["end"])

For ASR on GPU 0 and alignment on GPU 1, expose both GPUs, set CUDA_VISIBLE_DEVICES=0,1, and use --aligner-device cuda:1. vLLM uses its normal visible-device selection; the example uses one tensor-parallel ASR GPU. The same alignment API works with backend="transformers".

Important alignment boundaries:

• The helper conservatively limits a combined ASR+alignment call to 180 s, matching the installed upstream timestamp pipeline. For longer recordings, split at sensible speech boundaries, transcribe and align each segment, then add each segment's start offset to its word timestamps. It never silently truncates a 20-minute file to fit the aligner.
• Prefer verbatim text for acoustic alignment. Rewritten or omitted words in reading-optimised output need not have one-to-one spans in the audio.
• Nynorsk alignment is experimental; different written forms can affect timing. Do not align an English translation directly to Norwegian audio; the helper rejects this combination. Transcribe/align the spoken words first and handle translation timing as a separate task.
• Empty ASR outputs return an empty timestamp list. Forced alignment does not verify that a transcript is correct or prove that the words were spoken.

Long recordings and generation limits

Training includes short clips and long/sparse-speech examples up to 1,200 s (20 minutes). This is the supported input ceiling of the preparation/example path, not a guarantee of accurate 20-minute transcription. The small natural long-form evaluation cohort is much shorter than 20 minutes; long recordings need independent testing. Streaming behaviour has not been validated for this release helper.

The helper rejects audio over 1,200 s rather than cropping it. It disables processor truncation and defaults to 4,096 generated tokens. Dense long speech may require a larger token limit and context budget. If generation reaches the limit, it warns and returns truncated=true; do not mistake this for a complete transcript. Check memory and quality when increasing limits.

Training and data

This model was trained fresh from the custom-tokenizer Qwen base, not from the earlier Lunde 100k, longfix or hesitation-boost checkpoints. Those model cards informed these user instructions; their post-training results do not transfer to this release.

The corpus contains 23,350,245 target rows across 20 sources. Sources are shuffled by row epoch without oversampling to meet keyword targets. Different languages/styles can share audio; these are not 23 million independent recordings. The mix comprises:

• Consensus and Nemotron pseudo-labelled reading/verbatim data, with Bokmål, Nynorsk and English targets: 12 sources. Nynorsk/English targets are translated; pseudo-labels and translations can contain errors.
• Packed synthetic number and rare-word speech: two sources.
• Reconstructed Stortinget long reading recordings, dense six-way synthetic sessions and sparse six-way synthetic sessions: three sources. Six-way means three output languages multiplied by two styles; it does not mean six speakers.
• Recorded no-caption audio, the published 20% background-noise subset, and long noise-only audio: three sources, with intentionally empty transcripts. The full 10-million-row background-noise dataset is not additionally included.

Training uses each source's text field. For Norwegian targets, applicable sources use upstream NormForm normalisation, so spelling variants can differ from evaluation references. Keyword annotations populate SYSTEM context, not the target. Training manifests and audio are not distributed in this model repo. Exact source IDs/revisions are recorded in release_provenance.json.

Evaluation status and interpretation

The selected 28k evaluation was still queued when this release was prepared. No 24k, 0.10-run, Lunde or base-model scores should be attributed to the published 28k weights. Completed checkpoint evaluations and prediction reports are tracked in the internal W&B report run.

For context only, the earlier 24k checkpoint of the same .50 run had:

At that earlier checkpoint, reading-optimised noisy-NST WER was studio 2.58%, room_far 6.29%, Teams 5.56%, MELPe2400 4.51%, MELPe1200 8.14%. These are greedy corpus-level WERs, not means of sample WER. Short aggregates exclude NB-tale and Fleurs; noise and long cohorts are scored separately. Each short language cohort has 400 unique clips/four domains, noisy NST has 100 clips per condition, and the long cohort contains only seven recordings. The references are pinned; detailed revisions are in release provenance.

Empty-reference output is not necessarily hallucination. Earlier dashboards called any nonempty prediction on a no-caption reference a hallucination. Maintainer listening review of flagged 24k examples identified faint genuine speech/voices. Those examples must not be presented as confirmed invented speech. We have not silently changed the references, recalculated official metrics, or claimed that every output is grounded. Background-speech rejection and invented content require separate, audited evaluation.

Known limitations

• Rare runaway repetition, early stopping, missing words and incorrect names remain possible. A good average WER does not rule out severe individual errors.
• Verbatim hesitation coverage is incomplete; this model has not received the earlier Lunde checkpoint's dedicated hesitation-boost extension.
• Strong augmentation improved observed noise robustness but did not uniformly improve every language/style. This beta is not claimed to dominate the .10 run.
• Long-form support, timestamps, quiet speech, dialects, speaker groups, streaming and out-of-domain generalisation need additional validation. No comprehensive recording-level training/evaluation decontamination claim is made.
• Context hints can be ignored or misused. Do not use the system prompt to instruct the model to invent missing information.
• Output may include sensitive or offensive spoken content. Human review is required for consequential use; this is not a medical or legal decision system.

Internal reference and release contents

NB-ASR maintainers: exact selected training artifact

This private beta is checkpoint-28000, augmentation=0.50, from the fresh Bokfink02 0.6B production-comparison run, Slurm 2563783. Internal run name: NB-ASR-QWEN06-bokfink02-64gpu-32k-aug50-20261017. It is not the .10 model, not checkpoint-29624, and not a completed 32k run. The allocation timed out at approximately 29,624 updates; the maintainers selected the last full scheduled 28k checkpoint. The LR horizon was not compressed to 28k and no additional recovery/alignment fine-tuning was applied.

Weights, tokenizer, template and processor files are published byte-for-byte from the selected checkpoint. release_provenance.json records SHA-256 hashes and a source-revision inventory. inference.py and this model card are release support files, not modifications to the checkpoint. Optimizer/scheduler/RNG states, Trainer state, training arguments, logs, manifests, audio, transcripts and credentials are deliberately excluded. Full recovery state remains in internal storage.

Release verification covers checkpoint identity, tensor/tokenizer dimensions, prompt serialisation, package API/source compatibility, private Hub visibility and uploaded-file identity. The newly documented vLLM-plus-aligner path has not been executed end-to-end on a GPU as part of this publication; treat examples as integration starting points and verify on your deployment hardware.

References and acknowledgements

• Qwen3-ASR implementation and installation
• Original architecture
• Custom-tokenizer starting model
• Norwegian companion aligner
• Earlier Lunde 100k usage reference
• Earlier hesitation-focused usage reference

Developed within NB-ASR at the National Library of Norway, using the Qwen architecture and the work of the NB-ASR data, infrastructure and evaluation teams.