中文Sign up

aufklarer/Sidon-CoreML

🤗 Hugging Face sourceaudio-to-audiomit1.1 GBother✓ 8 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo aufklarer/Sidon-CoreML ./model-folder
Needs a seeder →

Sidon — Core ML (speech restoration / dereverberation)

On-device speech restoration (denoise + dereverberation + bandwidth restoration) for Apple Silicon, exported to Core ML (Neural Engine + GPU). Turns a noisy/reverberant clip into studio-quality 48 kHz speech — ideal for cleaning a voice-cloning reference before TTS, since it preserves speaker identity.

Two-stage pipeline:

16 kHz audio → [w2v-BERT log-mel front-end] → predictor (w2v-BERT 2.0, 8 layers)
            → cleansed features [1, T, 1024] → DAC decoder → 48 kHz audio

Variants

variant predictor vocoder bundle size peak RAM RTF (Core ML)
fp16 FP16 FP16 713 MB 1711 MB ~120×
int8 INT8 (k-means palettized) FP16 407 MB 1321 MB ~110×

Total 246 M params (predictor 193.6 M + DAC vocoder 52.4 M). Output sample rate 48 kHz. int8 keeps the vocoder at FP16 (audio quality); only the predictor is palettized.

Files

path description
fp16/Sidon-Predictor.mlpackage w2v-BERT 2.0 (8L) + merged LoRA → features (FP16)
fp16/Sidon-Vocoder.mlpackage DAC decoder → 48 kHz audio (FP16)
int8/Sidon-Predictor.mlpackage predictor, 8-bit palettized
int8/Sidon-Vocoder.mlpackage DAC decoder (FP16)

Quality (no-reference MOS, 10 s clip)

DNSMOS P.835 (SIG/BAK/OVRL, higher = better) and UTMOS (naturalness, 1–5):

audio SIG BAK OVRL UTMOS speaker cos
input (reverberant) 3.46 3.40 2.90 2.99 —
fp16 3.53 4.09 3.28 3.32 0.797
int8 3.54 4.11 3.29 3.23 0.796

Restoration lifts OVRL 2.90 → 3.29 (driven by BAK 3.40 → 4.11 — reverb removed). Quantization is near-lossless on DNSMOS and speaker similarity; UTMOS shows a small naturalness cost (fp16 −0.09, int8 −0.17). Numbers are a single clip — average over a set for a definitive figure.

Front-end

The graphs take input_features [1, T, 160] from the w2v-BERT 2.0 SeamlessM4T feature extractor (16 kHz input). The sequence length is fixed (T = 499 ≈ 10 s) — chunk longer audio in the runtime. The front-end and chunking are handled by speech-swift.

Usage

Use via the speech-swift Apple SDK, e.g.:

speech restore noisy-reference.wav -o clean.wav   # restore / dereverb on-device

Compute placement. The runtime places the two stages separately: the predictor on the Neural Engine, the DAC vocoder on the GPU, where its very wide convolutions load and run fastest. Override both with --compute-units ane|gpu|cpu|all, or per stage via predictorComputeUnits: / vocoderComputeUnits: in the Swift API.

// See speech-swift for the full API (loads the predictor + vocoder, runs the
// log-mel front-end, chunks, and writes 48 kHz audio).

Source

Exported from Sidon (sarulab-speech), checkpoint sidon-v0.1; paper arXiv:2509.17052. Base SSL encoder: facebook/w2v-bert-2.0; vocoder: DAC (descript-audio-codec). All components are MIT-licensed.

Links