wav2vec2-base-100h-repro

Paper: wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations (Baevski et al., 2020)
Official weights: facebook/wav2vec2-base-100h

Results

Greedy CTC decoding, no language model — comparable to the paper's "None" column in Appendix Table 9.

WER

	dev-clean	dev-other	test-clean	test-other
This model	6.3	15.1	6.5	15.4
No pre-training	12.6	35.8	12.7	38.1
Paper	6.1	13.5	6.1	13.3

Usage

import torch, torchaudio
from transformers import Wav2Vec2ForCTC, Wav2Vec2Processor

name = "nagohachi/wav2vec2-base-100h-repro"
processor = Wav2Vec2Processor.from_pretrained(name)
model = Wav2Vec2ForCTC.from_pretrained(name).eval()

waveform, sample_rate = torchaudio.load("utterance.flac")  # 16 kHz, mono
inputs = processor(waveform[0], sampling_rate=16_000, return_tensors="pt")

with torch.inference_mode():
    logits = model(inputs.input_values).logits

print(processor.batch_decode(logits.argmax(dim=-1)))

Training

Stage 1: self-supervised pre-training


Data	LibriSpeech `train-960`
Updates	400,000
Effective batch	5,750 s
Crop	250,000 samples (15.6 s)
Optimizer	Adam, β = (0.9, 0.98), ε = 1e-6, weight decay 0.01
Learning rate	5e-4
Precision	bf16 mixed

Stage 2: CTC fine-tuning


Data	LibriSpeech `train-clean-100`
Updates	50,000
Effective batch	1,335 s
Optimizer	Adam, β = (0.9, 0.98), no weight decay
Learning rate	2e-5
Model selection	best dev-other checkpoint

Loading…