CoreML / Apple Neural Engine build of Oriserve/Whisper-Hindi2Hinglish-Prime, packaged for WhisperKit so it can run fully on-device on Apple Silicon.
Converted for Speechmark, a private, on-device meeting notetaker for macOS, to transcribe code-mixed Hindi–English ("Hinglish") meetings without sending audio to a server.
This is a derivative work of Oriserve/Whisper-Hindi2Hinglish-Prime, which
is licensed under the Apache License 2.0. This conversion is released under the
same license. All model weights and their speech-recognition capability are the
work of Oriserve; the only changes here are
format conversion and inference-graph packaging (see below). The underlying
architecture derives from OpenAI's Whisper large-v3.
import WhisperKit
let config = WhisperKitConfig(
model: "Oriserve_Whisper-Hindi2Hinglish-Prime_889MB",
modelRepo: "nitinh/whisperkit-hinglish-coreml"
)
let pipe = try await WhisperKit(config)
// The model emits romanized Hinglish under the English token.
let options = DecodingOptions(task: .transcribe, language: "en")
let result = try await pipe.transcribe(audioPath: path, decodeOptions: options)
Added alignment_heads to generation_config.json. The upstream repo
omits this field. WhisperKit's TextDecoder expects a fourth output
(alignment_heads_weights), so conversion fails with
ValueError: Number of outputs provided, 4, do not match the number of outputs detected in the model, 3. The values were copied from
openai/whisper-large-v3, whose architecture is identical to this
fine-tune (32 encoder/decoder layers, 20 attention heads, d_model 1280,
vocab 51866 — verified field by field).
Converted to CoreML with
whisperkittools
(whisperkit-generate-model), producing MelSpectrogram.mlmodelc,
AudioEncoder.mlmodelc, and TextDecoder.mlmodelc.
Quantized to 4-bit mixed-bit palettization (group size 64), reducing the
bundle from 2.9 GB to 889 MB. Conversion fidelity: torch2coreml PSNR
54.5 (decoder) / 60.9 (encoder); all conversion tests passed.
No weights were retrained or pruned; the only numerical change is the quantization in step 3.
language: "en", matching the upstream model's documented behaviour.Oriserve_Whisper-Hindi2Hinglish-Prime/TextDecoder.mlmodelc/weights/weight.bin 1.8 GBOriserve_Whisper-Hindi2Hinglish-Prime/AudioEncoder.mlmodelc/weights/weight.bin 1.3 GBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/TextDecoder.mlmodelc/weights/weight.bin 558 MBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/AudioEncoder.mlmodelc/weights/weight.bin 322 MBOriserve_Whisper-Hindi2Hinglish-Prime/MelSpectrogram.mlmodelc/weights/weight.bin 373.4 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/MelSpectrogram.mlmodelc/weights/weight.bin 373.4 kBOriserve_Whisper-Hindi2Hinglish-Prime/TextDecoder.mlmodelc/coremldata.bin 0.7 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/TextDecoder.mlmodelc/coremldata.bin 0.7 kBOriserve_Whisper-Hindi2Hinglish-Prime/AudioEncoder.mlmodelc/coremldata.bin 0.4 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/AudioEncoder.mlmodelc/coremldata.bin 0.4 kBOriserve_Whisper-Hindi2Hinglish-Prime/MelSpectrogram.mlmodelc/coremldata.bin 0.4 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/MelSpectrogram.mlmodelc/coremldata.bin 0.4 kBOriserve_Whisper-Hindi2Hinglish-Prime/AudioEncoder.mlmodelc/analytics/coremldata.bin 0.2 kBOriserve_Whisper-Hindi2Hinglish-Prime/MelSpectrogram.mlmodelc/analytics/coremldata.bin 0.2 kBOriserve_Whisper-Hindi2Hinglish-Prime/TextDecoder.mlmodelc/analytics/coremldata.bin 0.2 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/AudioEncoder.mlmodelc/analytics/coremldata.bin 0.2 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/MelSpectrogram.mlmodelc/analytics/coremldata.bin 0.2 kBOriserve_Whisper-Hindi2Hinglish-Prime_889MB/TextDecoder.mlmodelc/analytics/coremldata.bin 0.2 kB有文件?提交种子。
提交种子