Model Card: Wav2vec_Classroom_WSP_FT
Model Overview
Model Name: Wav2vec_Classroom_WSP_FT
Version: 1.0
Developed By: Ahmed Adel Attia (University of Maryland)
Date: 2025
Description:
Wav2vec_Classroom_WSP_FT is an automatic speech recognition (ASR) model trained specifically for classroom speech transcription using a weakly supervised pretraining (WSP) approach. The model first undergoes supervised pretraining on weakly transcribed classroom data (NCTE-Weak) and is then fine-tuned using a small amount of human-verified gold-standard data (NCTE-Gold). This methodology allows the model to generalize well despite the scarcity of precisely transcribed classroom speech.
This model is adapted from Wav2vec-Classroom, which was trained using continued pretraining (CPT) on large-scale unlabeled classroom speech data. The adaptation involves further fine-tuning to leverage weak transcriptions before final refinement on high-quality annotations.
This model was originally trained using the fairseq library then ported into Huggingface.
The model should be run with n-gram LM beamsearch decoding for best results. We got our best results using this 5-gram LM we trained on classroom speech text.
Use Case:
- Speech-to-text transcription for classroom environments.
- Forced allignment of transcription with audio to provide character and word level boundaries.
- Educational research and analysis of classroom discourse.
- Low-resource ASR applications where gold-standard labels are limited.
Model Details
Architecture: Wav2vec2.0-based model fine-tuned with Fairseq
Training Data:
- NCTE-Weak: 5000 hours of weak transcriptions from the NCTE dataset.
- NCTE-Gold: 13 hours of manually transcribed classroom recordings.
Training Strategy:
- Weakly Supervised Pretraining (WSP): The model is first trained using NCTE-Weak transcripts, which contain alignment errors and omissions but provide useful weak supervision.
- Precise Fine-tuning: The pretrained model is fine-tuned on NCTE-Gold, ensuring it adapts to high-quality transcriptions.
Evaluation Results
Word Error Rate (WER) comparison on NCTE and MPT test sets:
| Training Data | NCTE WER | MPT WER |
|---|---|---|
| Baseline (TEDLIUM-trained ASR) | 55.82 / 50.56 | 55.11 / 50.50 |
| NCTE-Weak only | 36.23 / 32.30 | 50.84 / 46.09 |
| NCTE-Gold only | 21.12 / 16.47 | 31.52 / 27.93 |
| Self-training | 17.45 / 15.09 | 27.42 / 26.24 |
| NCTE-WSP-ASR (NCTE-Weak → NCTE-Gold) | 16.54 / 13.51 | 25.07 / 23.70 |
Limitations
- The model relies on weak supervision, and transcription quality is dependent on the balance between weak and gold-standard data.
- Classroom noise, overlapping speech, and spontaneous interactions may still lead to recognition errors.
- The model was trained specifically on elementary math classrooms and may not generalize well to other educational settings without further adaptation.
Usage Request
If you use the NCTE-WSP-ASR model in your research, please acknowledge this work and refer to the original paper submitted to Interspeech 2025.
For inquiries or collaborations, don't hesitate to contact me at aadel@umd.edu or ahmadadelattia@gmail.com