Loading…
Loading…
AI disclosure: This project was built with AI assistance (Claude, by Anthropic) for writing code and documentation. All creative and technical decisions (model design, experiments, evaluation) are mine, and I reviewed everything before publishing.
A causal, real-time speech denoiser for calls and meetings: 16 kHz, 10 ms algorithmic latency, 988k parameters, 0.15 ms of CPU per 5 ms block. The fourth version of my noise suppressor and the first to beat the noisy input on every metric.
Code, earlier versions and full write-up: GitHub: Audio_ML_Noise_Reduction
Trained on: the VoiceBank-DEMAND training set (CC BY 4.0) plus DNS Challenge clean speech mixed with DNS noise. Details below in Training data.
VoiceBank-DEMAND test set (824 files, 16 kHz), evaluated once with the best dev checkpoint.
| Metric | Noisy input | v2 | v3 | v4 |
|---|---|---|---|---|
| PESQ (WB) | 1.97 | 1.65 | 1.70 | 2.58 |
| STOI | 0.921 | 0.869 | 0.874 | 0.930 |
| SI-SDR (dB) | 8.45 | 13.36 | 13.17 | 17.21 |
| Algorithmic latency | 200 ms | 200 ms | 10 ms |
Streaming vs offline parity: 108.3 dB. CPU per 5 ms block (ONNX Runtime): 0.15 ms (real-time factor 0.03).
Fair-comparison note: v1-v3 never saw VoiceBank-DEMAND; v4 trains on its training split (different speakers from the test set), which is the standard protocol for this benchmark and part of the jump.
| File | What it is |
|---|---|
call_denoiser_v4.onnx |
One-frame streaming model: magnitude [1, 1, 257] + GRU state [2, 1, 256] in, mask + new state out |
call_denoiser_v4.pth |
PyTorch checkpoint (EMA weights, step 26,000) |
model.py |
Model definition and asymmetric-window STFT |
stream.py |
Real-time block engine (80-sample blocks in and out) |
# pip install onnxruntime numpy
from stream import StreamEngine # expects call_denoiser_v4.onnx next to it
eng = StreamEngine()
y_block = eng.process(x_block) # float32[80] at 16 kHz -> 80 denoised samples, 10 ms later
from stream import process_file # offline: whole mono 16 kHz array in, denoised array out
y = process_file(x)
Asymmetric analysis/synthesis windows (Wang et al. 2021): 32 ms analysis window, 10 ms synthesis window, 5 ms hop. Per frame: compressed magnitude -> Linear -> LayerNorm -> 2-layer GRU (256) -> sigmoid magnitude mask; noisy phase kept (v3 showed a learned phase hurt quality).
Training: 26,208 steps (batch 32, 4 s segments) on an RTX 2060. Compressed spectral loss (Braun & Tashev) + multi-resolution STFT; AdamW, warmup + cosine, EMA 0.999.
| Source | Amount | Use | License |
|---|---|---|---|
| VoiceBank-DEMAND training set (Valentini-Botinhao et al.): paired clean/noisy speech, 28 speakers | 11,572 utterances, about 9.4 h | 60% of training examples; speakers p226 and p287 held out for dev | CC BY 4.0 |
| DNS Challenge fullband clean speech (subset) | 13,733 clips, about 9.6 h | 40% of examples, mixed on the fly with DNS noise at SNR -5 to 20 dB; 5% of files held out for dev | Per source, see the DNS repo |
| DNS Challenge fullband noise (subset) | 8,000 clips, about 20.5 h | noise for the mixtures above | Per source, see the DNS repo |
| VoiceBank-DEMAND test set (824 utterances, 2 unseen speakers) | final evaluation only, never trained on or used to pick checkpoints | CC BY 4.0 |
Everything resampled to 16 kHz mono (from 48 kHz).