DKvideo voice gender classifier (int8)
Modified copy of
prithivMLmods/Common-Voice-Gender-Detection-IND-En-ONNX
(revision f354b22f2671b1d762b328d5776956c1f7124ac1, Apache-2.0).
Change: MatMul/Gemm weights quantised to int8 with onnxruntime
quantize_dynamic (361 MB → 122 MB). Labels and pre-processing are unchanged.
Used by the DKvideo app to tell male from female voices (how a voice sounds, not a person's gender) when choosing dubbing voices.
Benchmark (FLEURS, CC-BY 4.0)
1 070 clips: Vietnamese 461 male / 200 female, Mandarin 266 male / 143 female, each clean, cut to 2 s, and with pink noise at 5 dB SNR. Cells: accuracy on clips the method decided / share decided (p ≥ 0.8 or ≤ 0.2).
| clean | vi M | vi F | zh M | zh F | ms/clip |
|---|---|---|---|---|---|
| pitch (old) | 1.00 / 0.08 | 0.91 / 0.06 | 1.00 / 0.05 | 1.00 / 0.02 | 30 |
| ECAPA-TDNN (JaesungHuh) | 0.37 / 0.85 | 1.00 / 1.00 | 0.35 / 0.63 | 1.00 / 1.00 | 2245 |
| wav2vec2 | 0.80 / 0.99 | 1.00 / 1.00 | 1.00 / 1.00 | 1.00 / 1.00 | 1387 |
| pyannote embedding + LR | 1.00 / 0.29 | – / 0.00 | 1.00 / 0.50 | 1.00 / 0.87 | 280 |
| 2 s / noise 5 dB, wav2vec2 | vi M | vi F | zh M | zh F |
|---|---|---|---|---|
| 2 s | 0.74 / 1.00 | 1.00 / 1.00 | 0.99 / 1.00 | 1.00 / 1.00 |
| noise | 0.47 / 0.98 | 1.00 / 1.00 | 0.98 / 1.00 | 1.00 / 1.00 |
Findings that set the design:
- ECAPA calls most Asian male voices female (median p_male 0.1–0.3), so it was dropped.
- Most Vietnamese "male" clips that wav2vec2 calls female measure 155–200 Hz with pYIN and both models agree: FLEURS gender labels are noisy, so the real wav2vec2 error is lower than the table shows.
- The app's pitch tracker halves some female voices to ~130 Hz and noise pulls every voice under 120 Hz, so pitch may not overrule the model except for the very-high-pitch male check.
- int8 vs full precision on every 4th clip (268 per condition): same decision on 99.6 % (clean), 99.3 % (2 s), 98.9 % (noise); accuracy 0.918 / 0.888 / 0.776 vs 0.914 / 0.888 / 0.780. Speed is about the same (the conv front end stays float), the gain is size.