GE2E Speaker Verifier on 2.4 kb/s Coded Speech (V_coded, ICASSP 2018)

This is not an officially supported Google product.

*This model repository is an open-source reproduction of the paper W. Bastiaan Kleijn, Felicia S. C. Lim, Alejandro Luebs, Jan Skoglund, Florian Stimberg, Quan Wang, Thomas C. Walters, "WaveNet Based Low Rate Speech Coding," ICASSP 2018 (arXiv:1712.01120), created after the original publication using open-source frameworks and datasets.*

• GitHub Repository: wq2012/wavenet_coding
• PyPI Package: wavenet-coding
• Interactive Demo Space: wq2012/wavenet-coding-demo

Model Summary

3-layer Projected-LSTM (LSTMP) text-independent speaker verification model (128 LSTM units projected to 64 dimensions per layer over 40-channel frame-level log-Mel filterbank features, followed by L2 normalization of the final recurrent frame state) trained with Generalized End-to-End (GE2E) softmax loss on 2.4 kb/s Parametric WaveNet coded speech (Section 3.4 of Kleijn et al., ICASSP 2018).

Exported Artifacts Included in This Repository

1. model.safetensors: Hugging Face SafeTensors FP32 weights.
2. config.json: Complete model architecture specification (num_lstm_layers=3, lstm_units=128, projection_dim=64, n_mels=40).
3. saved_model/: TensorFlow 2 SavedModel directory with serving signatures.
4. model_fp32.tflite (1,157,512 bytes) & model_int8.tflite (558,856 bytes): Verified TensorFlow Lite flatbuffers.
5. model_f16.gguf & model_q4_k_m.gguf: Verified GGUF v3 binary files (15 tensors) for C++/edge inference.
6. training_metrics.json & evaluation_metrics.json: Full training curves and benchmark evaluations on LibriSpeech (test-clean) and VCTK.

Training Progression (training_metrics.json)

• Training Corpus: 40 speakers (80 utterances coded at 2.4 kb/s, 158.06 s audio) from LibriSpeech test-clean + CSTR VCTK; evaluated on held-out speakers
• Wall-Clock Training Time: 339.11 s (40 epochs)
• Initial GE2E Loss / Held-Out EER: 2.3694 (25.000% eval EER, 29.793% epoch-1 train EER)
• Final GE2E Loss / Held-Out EER: 0.5530 (12.946% eval EER, 9.127% train EER)

Evaluation Results — Speaker Verification on 2.4 kb/s Coded Speech (Section 3.4)

Usage Examples

Load SafeTensors & Extract L2-Normalized Speaker Embeddings (wavenet-coding)

import json
import os
import numpy as np
from huggingface_hub import snapshot_download
from wavenet_coding import configs, dataset, export, model

model_dir = snapshot_download("wq2012/wavenet-speaker-verifier-coded")
with open(os.path.join(model_dir, "config.json"), "r", encoding="utf-8") as f:
    cfg = configs.SpeakerVerifierConfig.from_dict(json.load(f))
verifier = model.build_speaker_verifier(cfg)
export.load_safetensors_into_model(
    verifier, os.path.join(model_dir, "model.safetensors")
)

t = np.linspace(0.0, 0.5, 8000, endpoint=False, dtype=np.float32)
waveform_16k = 0.12 * np.sin(2.0 * np.pi * 180.0 * t)
mel_feats = dataset.extract_log_mel_features(waveform_16k, max_frames=80)
embedding = verifier(mel_feats[np.newaxis, ...], training=False).numpy()[0]
print("Embedding shape:", embedding.shape, "L2 norm:", np.linalg.norm(embedding))

Citation

@inproceedings{kleijn2018wavenet,
  title={WaveNet Based Low Rate Speech Coding},
  author={Kleijn, W. Bastiaan and Lim, Felicia S. C. and Luebs, Alejandro and Skoglund, Jan and Stimberg, Florian and Wang, Quan and Walters, Thomas C.},
  booktitle={2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={676--680},
  year={2018},
  organization={IEEE}
}