Whistle — GGUF for the opencoti engine

Cactus Compute's Whistle (Apache-2.0) as one 17 MB GGUF file: a speech-to-text model for seven languages (English, German, French, Spanish, Italian, Dutch, Polish) with language detection, word timestamps with probabilities, and live transcription. It runs on the CPU (and on Metal on a Mac); it takes no GPU memory away from a language model loaded beside it.

ggml-silero-v6.2.0.bin is the optional speech detector: Silero VAD v6.2.0 (MIT) in the ggml conversion published by ggml-org/whisper-vad, byte for byte, mirrored here so the model and its detector come from one place.

The weights are Cactus's own, byte for byte as stored in whistle.cact (revision ca528760): the 2-bit and 4-bit Cactus Quants tensors are kept as they are (ggml tensor types CQ2_0 / CQ4_0) with the fp16 remainder. Nothing was re-quantised.

What runs it

The file needs an engine that knows the whistle architecture and the two Cactus Quants tensor types: opencoti llamafile c12 or newer (and xOllama releases built on it). A stock whisper.cpp / transcribe.cpp / llama.cpp build does not load it.

opencoti-llamafile --server -m <your language model>.gguf --stt-model whistle.gguf

• POST /v1/audio/transcriptions — the OpenAI route: multipart file, optional language, response_format json | text | verbose_json | srt | vtt; timestamp_granularities[]=word gives words: [{word, start, end, probability}].
• stream=true on the same route — server-sent events with the text as it is recognised.
• /v1/realtime — an OpenAI Realtime transcription session over a websocket, for live audio.
• Optional speech detector in front of the model: --stt-vad-model ggml-silero-v6.2.0.bin (the file in this repository; audio without speech returns empty text instead of a guess).

Audio longer than 30 seconds is cut into windows at sentence ends, the language is detected per window. Translation is not supported (/v1/audio/translations answers 501).

Measured

On 30 clips from FLEURS (dev split, the seven languages, 827 reference words), scored against the human transcripts: word error rate 17.41 % for this port and 17.65 % for the Cactus reference engine running the same weights; 571 of 573 words land on the same 80 ms frame as the reference. With the speech detector in front: 16.08 %, and 24 of 24 non-speech recordings (music, traffic, rain and the like) return empty text.

Credits and licence

Model, training and the Cactus Quants format: Cactus Compute. This repository only repackages the published weights as GGUF. Apache-2.0, as the original; the LICENSE file is the original's.

Speech detector: Silero Team, MIT (LICENSE.silero-vad); ggml conversion by the whisper.cpp project (ggml-org/whisper-vad).