Whisper small, CTranslate2 int8_float16
Whisper small converted for faster-whisper
/ CTranslate2 with int8 weights and float16 activations — the computation faster-whisper did at every load with
compute_type="int8_float16" from a float16 conversion, done once. Used by the myDubber local backend and server
for speech recognition.
Built from openai/whisper-small at commit 973afd24965f72e36ca33b3055d56a652f456b4d by
model_builds/whisper.py
(dubber): CTranslate2's transformers converter with quantization="int8_float16" (ctranslate2
4.8.3, transformers 5.17.0), the weights loaded as float16 first and the
encoder's two convolutions kept float16 — exactly what CTranslate2 computes when it loads a float16 conversion with
compute_type="int8_float16" (it never quantizes convolutions at load); tokenizer.json and
preprocessor_config.json copied from the source. The weights are modified: converted to CTranslate2's layout
and the linear weights quantized to int8 (per-row absmax, CTranslate2's own); nothing was trained.
from faster_whisper import WhisperModel
model = WhisperModel("<this repo>", device="cuda", compute_type="int8_float16")
model.bin: 0.23 GB, sha256 8d26a9e58c09fd26c70d33bbb20abcb90d7e089cc19e8a5123d9ef7ea2e82b86
Licence
Apache-2.0, as the source card declares (copy in LICENSE).
Copyright of the weights: OpenAI.