Moonshine Streaming Tiny Arabic (ONNX & INT8 Quantized)

This repository contains the ONNX-converted and dynamically quantized (INT8) weights for moonshine-ai/moonshine-streaming-tiny-ar, optimized for ultra-low latency browser & edge speech recognition.

Model Details

• Base Architecture: Moonshine Streaming (Encoder-Decoder with sliding window attention)
• Parameters: 27 Million
• Vocab Size: 12,288 (Arabic)
• Audio Sample Rate: 16,000 Hz (mono)
• Frame Rate: 5 ms (320 samples per frame)

Files Included

Performance

• Whisper Tiny Latency: ~2,000 ms per phrase
• Moonshine Tiny ONNX Latency: ~15–25 ms per phrase (~100x faster)
• Total Download Size (Quantized): ~28 MB

Usage with ONNX Runtime (Python)

import onnxruntime as ort
import numpy as np
from transformers import AutoTokenizer

# 1. Load sessions
enc_session = ort.InferenceSession("encoder_model_quantized.onnx")
dec_session = ort.InferenceSession("decoder_model_quantized.onnx")
tokenizer = AutoTokenizer.from_pretrained(".")

# 2. Encode audio (16kHz float32 mono)
audio = np.random.randn(1, 16000).astype(np.float32)
enc_out = enc_session.run(None, {"input_values": audio})[0]

# 3. Autoregressive decode
tokens = [1]  # BOS token
for _ in range(50):
    input_ids = np.array([tokens], dtype=np.int64)
    logits = dec_session.run(None, {
        "input_ids": input_ids,
        "encoder_hidden_states": enc_out
    })[0]
    next_token = int(np.argmax(logits[0, -1]))
    if next_token in (0, 2):  # PAD or EOS
        break
    tokens.append(next_token)

print("Transcription:", tokenizer.decode(tokens[1:], skip_special_tokens=True))

Usage in JavaScript / Browser

import * as ort from 'onnxruntime-web';
import { AutoTokenizer } from '@huggingface/transformers';

const encoder = await ort.InferenceSession.create('encoder_model_quantized.onnx');
const decoder = await ort.InferenceSession.create('decoder_model_quantized.onnx');
const tokenizer = await AutoTokenizer.from_pretrained('Sharjeelbaig/moonshine-streaming-tiny-ar-onnx');