This repository contains an MLX-native int8 conversion of Cohere Transcribe for local automatic speech recognition on Apple Silicon. Eligible Linear and Embedding weights use affine 8-bit quantization with group size 64; convolution and normalization layers retain their native floating-point precision.
It is intended for local transcription with mlx-speech, without a PyTorch runtime or cloud API dependency at inference time.
| Path | Precision |
|---|---|
mlx-int8/ |
int8 quantized weights |
cohere-transcribe-03-2026Command-line transcription with mlx-speech:
python scripts/generate/cohere_asr.py \
--audio input.wav \
--output transcript.txt
Minimal Python usage:
import numpy as np
import soundfile as sf
from mlx_speech.generation import CohereAsrModel
audio, sr = sf.read("input.wav", dtype="float32", always_2d=False)
if audio.ndim > 1:
audio = audio.mean(axis=1)
if sr != 16000:
old_len = len(audio)
new_len = int(round(old_len * 16000 / sr))
audio = np.interp(np.linspace(0, old_len - 1, new_len), np.arange(old_len), audio).astype(np.float32)
model = CohereAsrModel.from_path("mlx-int8")
result = model.transcribe(audio, sample_rate=16000, language="en")
print(result.text)
scales and biases tensors.transcribe(), which matches the runtime requirement.Apache 2.0 — following the upstream Cohere Transcribe model license. Check the original Cohere release for current terms.
CohereLabs/cohere-transcribe-03-2026