WaveNet Closed-Loop Waveform Speech Coder (WW, ICASSP 2018)
This is not an officially supported Google product.
*This model repository is an open-source reproduction of the paper W. Bastiaan Kleijn, Felicia S. C. Lim, Alejandro Luebs, Jan Skoglund, Florian Stimberg, Quan Wang, Thomas C. Walters, "WaveNet Based Low Rate Speech Coding," ICASSP 2018 (arXiv:1712.01120), created after the original publication using open-source frameworks and datasets.*
- GitHub Repository: wq2012/wavenet_coding
- PyPI Package:
wavenet-coding - Interactive Demo Space: wq2012/wavenet-coding-demo
Model Summary
Unconditioned closed-loop autoregressive WaveNet speech coder (WW, Section 2.3 & Section 3.2) predicting 256-level 8-bit μ-law categorical posteriors at 16 kHz (2 blocks × 10 dilated causal residual layers) paired with a bit-exact 32-bit integer arithmetic coder.
Exported Artifacts Included in This Repository
model.safetensors: Hugging Face SafeTensors FP32 weights.config.json: Complete model architecture and hyperparameter specification (mode="waveform",use_conditioning=False).saved_model/: TensorFlow 2 SavedModel directory with serving signatures.model_fp32.tflite(1,114,152bytes) &model_int8.tflite(238,392bytes): Verified TensorFlow Lite flatbuffers.model_f16.gguf&model_q4_k_m.gguf: Verified GGUF v3 binary files (164tensors) for C++/edge inference.training_metrics.json&evaluation_metrics.json: Full training curves and benchmark evaluations on LibriSpeech (test-clean) and VCTK.
Training Progression (training_metrics.json)
- Training Corpus:
48speaker-disjoint utterances (64windows,105.82 saudio) from LibriSpeechtest-clean+ CSTR VCTK at 16 kHz (-18.4 dBFSactive level) - Wall-Clock Training Time:
35.48 s(15epochs) - Initial Eval Cross-Entropy:
7.8660bits/sample (5.4523nats/sample) - Final Eval Cross-Entropy:
4.8773bits/sample (3.3807nats/sample, train:5.4709bits/sample)
Evaluation Results — Closed-Loop Waveform Coding Rates (Section 3.2)
| Dataset / Split | Mean Conditional Entropy H̄ (bits/sample) | Theoretical Lower-Bound Bitrate (kb/s) | Cross-Entropy Rate R (bits/sample) | Cross-Entropy Bitrate (kb/s) | Active Arithmetic Coded Rate (bits/sample) | Active Arithmetic Bitrate (kb/s) | Arithmetic Coder Lossless Match Rate |
|---|---|---|---|---|---|---|---|
| Paper Reported (WSJ0, Sec. 3.2, 60-layer conditioned WW) | 2.61 | ~42.0 | 2.62 (2.65 Eq. 3 gen. rate) | ~42.0 | — | — | — |
Reproduced — LibriSpeech (test-clean, 16 utt) |
6.0642 | 97.03 | 5.5285 (3.8320 nats) | 88.46 | 5.5230 | 88.37 | 100.0% (16/16) |
| Reproduced — VCTK Corpus (16 utt) | 6.0231 | 96.37 | 5.3500 (3.7083 nats) | 85.60 | 5.3485 | 85.58 | 100.0% (16/16) |
Waveform Quality Comparison (WW vs. 8-bit μ-law & 16 kHz Reference)
| Dataset | Codec | Bitrate (kb/s) | ITU-T P.862.2 PESQ-WB ↑ | STOI (0–1) ↑ | F0 RMSE (Hz) ↓ | MCD (dB) ↓ | LSD (dB) ↓ | SegSNR (dB) ↑ | High-Band (4–8 kHz) Energy (%) |
|---|---|---|---|---|---|---|---|---|---|
LibriSpeech (test-clean) |
Reference (16 kHz) | 256.00 | 4.644 | 1.000 | 0.000 | 0.000 | 0.000 | 45.000 | 5.378% |
LibriSpeech (test-clean) |
8-bit μ-law (128 kb/s) | 128.00 | 4.141 | 0.999 | 0.103 | 0.470 | 0.870 | 35.144 | 5.388% |
LibriSpeech (test-clean) |
WaveNet Waveform (WW) |
88.46 (88.37 active) | 4.141 | 0.999 | 0.103 | 0.470 | 0.870 | 35.144 | 5.388% |
| VCTK Corpus | Reference (16 kHz) | 256.00 | 4.644 | 1.000 | 0.000 | 0.000 | 0.000 | 45.000 | 1.604% |
| VCTK Corpus | 8-bit μ-law (128 kb/s) | 128.00 | 3.721 | 0.999 | 0.121 | 0.552 | 0.860 | 34.748 | 1.613% |
| VCTK Corpus | WaveNet Waveform (WW) |
85.60 (85.58 active) | 3.721 | 0.999 | 0.121 | 0.552 | 0.860 | 34.748 | 1.613% |
Usage Examples
Load SafeTensors in Python (wavenet-coding)
import numpy as np
from huggingface_hub import snapshot_download
from wavenet_coding import inference
model_dir = snapshot_download("wq2012/wavenet-waveform-coder")
codec = inference.WaveNetWaveformCodec.from_pretrained(model_dir)
t = np.linspace(0.0, 0.2, 3200, endpoint=False, dtype=np.float32)
waveform_16k = 0.12 * np.sin(2.0 * np.pi * 180.0 * t)
result = codec.encode_and_decode(waveform_16k, run_arithmetic_coder=True)
print("Lossless exact match:", result["lossless_exact_match"])
print("Active arithmetic rate (kb/s):", result["arithmetic_kbps_active"])
Run TFLite Flatbuffer (model_int8.tflite)
import tensorflow as tf
interpreter = tf.lite.Interpreter(model_path="model_int8.tflite")
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
Inspect GGUF v3 Binary (model_q4_k_m.gguf)
from wavenet_coding import export
header = export.inspect_gguf_header("model_q4_k_m.gguf")
print(header)
Citation
@inproceedings{kleijn2018wavenet,
title={WaveNet Based Low Rate Speech Coding},
author={Kleijn, W. Bastiaan and Lim, Felicia S. C. and Luebs, Alejandro and Skoglund, Jan and Stimberg, Florian and Wang, Quan and Walters, Thomas C.},
booktitle={2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
pages={676--680},
year={2018},
organization={IEEE}
}