Kim Mel-Band RoFormer (vocals) on Core ML
KimberleyJSN/melbandroformer (MIT, 228 M parameters) as a float16 Core ML .mlpackage, the vocal model behind slurper. Runs on the GPU on macOS 15 and later. The instrumental is the mix minus the vocals.
Graph
One 8 s chunk at 44.1 kHz (352 800 samples), with the STFT and inverse STFT folded into the graph as constant DFT matmuls, the periodic Hann window included:
| name | shape | meaning |
|---|---|---|
frames (input) |
[1, 2, 801, 2048] |
unwindowed 2048-sample frames at hop 441 of the chunk reflect-padded by 1024 on each side |
recon (output) |
[1, 2, 801, 2048] |
windowed inverse-DFT frames of the vocals |
The host overlap-adds recon at hop 441, divides by the summed squared window, and drops the 1024-sample pad. Chunks across a song overlap by half with linear crossfades. Sources/SlurperKit/VocalSeparator.swift in slurper is a reference host.
Conversion
scripts/convert_melband_roformer.py in slurper, after the coreai-model-zoo recipe: the band-average scatter becomes a constant matmul, the complex mask multiply becomes real arithmetic, rotary angles become constants, and attention goes through scaled_dot_product_attention. torch.jit.trace, then coremltools 9.0 to an ML program with float16 precision. Against the PyTorch model on the golden chunk: the re-authored float32 graph at cosine 0.9999984, the Core ML float16 model at cosine 0.999946 (39.6 dB SDR).
Files
mbr_fp16.mlpackage/Manifest.jsonmbr_fp16.mlpackage/Data/com.apple.CoreML/model.mlmodelmbr_fp16.mlpackage/Data/com.apple.CoreML/weights/weight.bin(490 MB)golden_raw.f32: an 8 s stereo excerpt with vocals, channel-major float32, first published with the Core AI conversion (mlboydaisuke/MelBandRoformer-Vocal-CoreAI);scripts/convert_melband_roformer.pynow reads it from this repositorygolden_vocals.f32: PyTorch's vocals for it, same layout, for host self-tests
Attribution
Mel-Band RoFormer by Ju-Chiang Wang, Wei-Tsung Lu and Minz Won (ByteDance). Checkpoint by Kimberley Jensen, trained with ZFTurbo's training code on lucidrains's BS-RoFormer implementation. Export recipe from coreai-model-zoo (Daisuke Majima, BSD-3-Clause). This conversion is MIT.