whisper-base-onnx

Transformers.js-compatible ONNX conversion of openai/whisper-base. The encoder and decoder export attention tensors, and the original Whisper alignment heads are preserved for word-level timestamps.

Usage

import { pipeline } from "@huggingface/transformers";

const transcriber = await pipeline(
  "automatic-speech-recognition",
  "Sharjeelbaig/whisper-base-onnx",
  { dtype: "q8" },
);

const result = await transcriber(audioFloat32Array, {
  language: "ar",
  task: "transcribe",
  return_timestamps: "word",
});

The repository contains fp32 and ARM64 dynamically quantized q8 encoder and merged decoder graphs. q8 is recommended for local CPU inference; use dtype: "fp32" when the unquantized graphs are preferred.

Files

The model files are under onnx/, following the Transformers.js layout:

• encoder_model.onnx and decoder_model_merged.onnx — fp32 graphs
• encoder_model_quantized.onnx and decoder_model_merged_quantized.onnx — q8 graphs

Conversion

convert-model.py is the reproducible development-only exporter. It enables eager attention and explicitly exposes encoder, decoder, and cross-attention outputs before the cached and non-cached decoder graphs are merged.

python convert-model.py --model openai/whisper-base --output export
optimum-cli onnxruntime quantize \
  --onnx_model export --arm64 --per_channel -o quantized

The source checkpoint is Apache-2.0 licensed. See the upstream model card for training details, evaluation results, and limitations.