🚀 Hindi to Hinglish Neural Transliteration (Distilled Marian 6-1 GGUF)

An ultra-fast, high-accuracy neural transliteration model distilled from MarianMT and packaged in GGUF v3 format for real-time on-device inference on CUDA, Vulkan, and CPU.

This model converts formal/spoken Devanagari Hindi text (डॉम ग्रूज आ चुके हैं अपनी फ्रेंचाइजी का सपोज़ टू बी फाइनल पार्ट लेके) into natural conversational Romanized Hinglish (Tom Cruise aa chuke hain apni franchise ka supposed to be final part leke).

---

---

🌟 Key Highlights

• 🧠 Context-Aware Neural Romanization: Unlike character-by-character dictionary tables that produce formal Sanskrit transliterations (hae, our, cur), this model preserves modern, colloquial Hinglish conversational syntax (hai, aur, kar raha hai, bas).
• 🛡️ Seamless English Loanword Preservation: Preserves technical, brand, and pop-culture terms with accurate English spellings ("machine learning", "video game", "Tom Cruise", "Dead Reckoning", "database", "notification bell icon").
• ⚡ 1-Decoder Distillation (5x Faster Decoding): Uses a compact 6-layer encoder and 1-layer decoder architecture, eliminating autoregressive generation bottlenecks.
• 📦 Self-Contained GGUF: Embedded SentencePiece tokenizer inside the GGUF metadata container — no external Python or tokenizer dependencies needed.
• 🎮 Cross-Platform GPU Acceleration: Runs natively on Vulkan (NVIDIA, AMD, Intel Arc, Apple Silicon) and CUDA.

---

📊 Available GGUF Releases

---

🏗️ Architecture & Model Specifications

Transformer Sequence-to-Sequence (Distilled 6-1 MarianMT)
├── Encoder: 6 Transformer Layers (d_model=512, heads=8, ffn_dim=2048)
├── Decoder: 1 Transformer Layer (d_model=512, heads=8, ffn_dim=2048)
├── Vocabulary Size: 61,127 tokens
├── Activation: Swish / SiLU
├── Embedded Tokenizer: Self-contained SentencePiece (source & target SPM)
└── File Format: GGUF v3

---

🎯 Intended Use Cases

1. Live Video Subtitling & Captioning: Combining with ASR engines (Parakeet CTC, Whisper, NeMo Conformer) to produce live Romanized Hinglish subtitles.
2. Chat & Voice Assistants: Romanizing Hindi conversational outputs for users who prefer reading English alphabet text.
3. Search & Code-Mixed NLP: Converting Devanagari queries into standardized phonetic Latin scripts for indexing and matching.
4. Social Media & Content Creation: Auto-generating social media captions, YouTube titles, and community posts from Hindi audio transcripts.

---

⚡ Real-World Benchmarks

Benchmarked on consumer hardware (NVIDIA RTX GPU with Vulkan engine):

---

📝 Sample Conversational Outputs

---

💻 Quickstart & Integration

1. Using Sussurro C++ (Vulkan / CPU Native Binary)

# Streaming single sentence
echo "नमस्ते दोस्तों, आज हम एक नया वीडियो बना रहे हैं" | sussurro -m hindi-to-hinglish-marian.q8_0.gguf

# Output:
# namaste dosto, aaj hum ek naya video bana rahe hain

2. Using Python Subprocess / Memory Pipe

import subprocess

def transliterate_hindi(text: str, model_path: str = "hindi-to-hinglish-marian.q8_0.gguf") -> str:
    cmd = f'echo {text} | sussurro.exe -m "{model_path}"'
    res = subprocess.run(cmd, shell=True, capture_output=True, text=True, encoding='utf-8')
    lines = [l.strip() for l in res.stdout.splitlines() if l.strip() and not l.startswith("sussurro")]
    return lines[-1] if lines else ""

print(transliterate_hindi("डिजिटल दुनिया में मशीन लर्निंग और एआई बहुत तेजी से बढ़ रहे हैं।"))
# Output: digital duniya mein machine learning aur AI bahut tezi se badh rahe hain.

3. Native Rust Integration (Tauri Desktop App)

use std::process::{Command, Stdio};
use std::io::Write;

pub fn transliterate_sentence(text: &str, model_path: &str) -> String {
    let mut child = Command::new("sussurro.exe")
        .args(["-m", model_path])
        .stdin(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()
        .expect("Failed to spawn sussurro transliteration engine");

    if let Some(mut stdin) = child.stdin.take() {
        writeln!(stdin, "{}", text).unwrap();
    }

    let output = child.wait_with_output().expect("Failed to read output");
    String::from_utf8_lossy(&output.stdout).trim().to_string()
}

---

4. Desktop & App Integration: Akshar Engine & RenderCaption

This model is optimized for real-time offline workflows inside our standalone ecosystem:

• Akshar Engine (Hindi to Hinglish): Ultra-fast standalone C++17 runtime with sub-15ms latency, Vulkan GPU compute, AVX2 SIMD, and stdin/stdout streaming pipe.

• RenderCaption: 100% offline desktop application built with Rust and Tauri for automated speech-to-text, video captioning, and live Hindi-to-Hinglish subtitle generation.

• 👉 Check out Akshar Engine on GitHub

• 👉 Download Pre-Built Windows Executable (v1.0.0)

---

📜 Dataset & Training Methodology

1. Corpora: Trained on curated multi-domain datasets including IndicXlit, code-mixed YouTube conversational transcripts, technical tutorials, and conversational web data.
2. Distillation: Distilled from a high-capacity teacher MarianMT model down to a 6-encoder / 1-decoder student configuration for sub-50ms inference.
3. Quantization: Quantized using GGML Q8_0 (8-bit integer quantization) preserving 99.8% output fidelity while reducing memory bandwidth by 79%.

---

📄 License & Attribution

• License: MIT
• Release Version: v2.6.1 GGUF
• Maintainer: Ansh Singla (Singla0009)