DAW Vocal Denoiser

AI disclosure: This project was built with AI assistance (Claude, by Anthropic) for writing code and documentation. All creative and technical decisions (model design, training data, evaluation) are mine, and I reviewed everything before publishing.

A causal, real-time vocal denoiser for tracking vocals in a DAW: 48 kHz, 10 ms algorithmic latency, 1.25M parameters, 0.19 ms of CPU per 5 ms block. It targets the noise a home studio actually has: room tone, hiss, mains hum and headphone bleed, and it is built to leave clean takes alone.

Code, training recipe and full write-up: GitHub: Daw-Vocal-Denoiser

Trained on: about 39 minutes of my own vocal takes, the VocalSet singing dataset (CC BY 4.0) and DNS Challenge clean speech + noise, with simulated headphone bleed from my own instrumentals. Details below in Training data.

Results

Held-out singing never used for training or model selection (a held-out song of mine plus held-out VocalSet singers), mixed with dev-split noise at 0-40 dB SNR, run through the streaming engine. 200 items.

Files

Usage

# pip install onnxruntime numpy
from stream import StreamEngine          # expects daw_denoiser.onnx next to it
eng = StreamEngine()
y_block = eng.process(x_block)           # float32[240] at 48 kHz -> 240 denoised samples, 10 ms later

from stream import process_file          # offline: whole mono 48 kHz array in, denoised array out
y = process_file(x)

Model

Asymmetric analysis/synthesis windows (Wang et al. 2021) give long-window frequency resolution (31 Hz bins, enough for low sung harmonics) at short-window latency. Per 5 ms frame: compressed magnitude -> Linear -> LayerNorm -> 2-layer GRU (256) -> sigmoid magnitude mask. The noisy phase is kept.

Training: 108,773 steps (batch 32, 4 s segments) on an RTX 4060; best dev checkpoint at step 77,000. Compressed spectral loss (Braun & Tashev) + multi-resolution STFT; AdamW, warmup + cosine, EMA 0.999.

Training data

Clean targets are drawn 35% own vocals / 35% VocalSet / 30% DNS speech; every example is mixed on the fly with noise.

Mixing: SNR 0 to 40 dB, 10% clean-only examples (so clean takes pass through untouched), 30% get a synthetic room applied to input and target alike. 48 kHz mono throughout. No commercial sample packs (songs using Cymatics samples were left out of the bleed set).

Intended use and limits

• For monitoring and cleaning vocal takes (singing or speech) recorded in untreated rooms.
• Removes noise, not reverb: rooms were applied to input and target alike, so the room stays.
• Not tested on heavy-distortion or extreme-SNR (below 0 dB) material; very loud bleed can survive.
• Mono, 48 kHz. Resample other rates first.