whisper-tiny-it-ct2-int8

CTranslate2 INT8 quantized version of LocalAI-io/whisper-tiny-it for fast CPU inference.

Author: Ettore Di Giacinto

Brought to you by the LocalAI team. This model can be used directly with LocalAI.

Usage with LocalAI

This model is ready to use with LocalAI via the whisperx backend.

Save the following as whisperx-tiny-it.yaml in your LocalAI models directory:

name: whisperx-tiny-it
backend: whisperx
known_usecases:
  - transcript
parameters:
  model: LocalAI-io/whisper-tiny-it-ct2-int8
  language: it

Then transcribe audio via the OpenAI-compatible endpoint:

curl http://localhost:8080/v1/audio/transcriptions \
  -H "Content-Type: multipart/form-data" \
  -F file="@audio.mp3" \
  -F model="whisperx-tiny-it"

Model Details

Base model: openai/whisper-tiny, fine-tuned on Common Voice 25.0 Italian
Quantization: INT8 via CTranslate2
Size: 39MB (vs 578MB for the original fp32 model)
WER: 27.1% on Common Voice 25.0 Italian test set

Usage

faster-whisper

from faster_whisper import WhisperModel

model = WhisperModel("LocalAI-io/whisper-tiny-it-ct2-int8", device="cpu", compute_type="int8")
segments, info = model.transcribe("audio.mp3", language="it")
for segment in segments:
    print(f"[{segment.start:.1f}s - {segment.end:.1f}s] {segment.text}")

WhisperX

import whisperx

model = whisperx.load_model("LocalAI-io/whisper-tiny-it-ct2-int8", device="cpu", compute_type="int8")
result = model.transcribe("audio.mp3", language="it")

LocalAI

This model is compatible with LocalAI for local, self-hosted AI inference.

LocalAI-io/whisper-tiny-it-ct2-int8