htdemucs on Core ML
htdemucs (Meta, MIT) as a Core ML .mlpackage, the drums, bass and other model behind slurper. Runs on the GPU on macOS 15 and later.
Graph
The model's real-valued core for one 7.8 s segment at 44.1 kHz, HTDemucs.forward from the normalization to just before _mask, in float32 because float16 overflows:
| name | shape | meaning |
|---|---|---|
mix (input) |
[1, 2, 343980] |
stereo segment |
spec (input) |
[1, 4, 2048, 336] |
_magnitude(_spec(mix)): left real, left imaginary, right real, right imaginary |
time (output) |
[1, 8, 343980] |
denormalized time branch, [source, channel] |
freq (output) |
[1, 16, 2048, 336] |
denormalized frequency branch, [source, channel, real/imaginary] |
Sources are drums, bass, other, vocals. The host computes the STFT (_spec: 4096-point periodic Hann, hop 1024, reflect padding, normalized), inverts freq with _ispec, adds time, and crossfades segments overlapping by a quarter as apply_model does. Sources/SlurperKit/Demucs.swift in slurper is a reference host.
Conversion
scripts/convert_htdemucs.py in slurper: torch.jit.trace of the core with the attention fast path disabled, then coremltools 9.0 to an ML program with float32 precision. Before saving, the Core ML output on the GPU is compared with PyTorch on a synthetic segment: drums 114 dB, bass 128 dB, other 114 dB, vocals 100 dB SDR.
Files
htdemucs_fp32.mlpackage/Manifest.jsonhtdemucs_fp32.mlpackage/Data/com.apple.CoreML/model.mlmodelhtdemucs_fp32.mlpackage/Data/com.apple.CoreML/weights/weight.bin(209 MB)
Attribution
Hybrid Transformer Demucs by Simon Rouard, Francisco Massa and Alexandre Défossez (Meta AI), MIT. This conversion is MIT as well.