tiny-superfast-agentic-MLM-6.5m-v1: a 6.4M-parameter voice-command parser for laptops and Raspberry Pi

tiny-superfast-agentic-MLM-6.5m-v1 turns one short spoken or typed command into a structured command that a program can execute. It understands English and Hinglish (Roman-script Hindi-English), with some Devanagari Hindi, and covers 371 actions (audio, display, Wi-Fi, apps, files, timers, power, network, services, Raspberry Pi GPIO/I2C and more), and runs on a plain CPU with NumPy only: no GPU, no PyTorch and no internet needed at inference.

ENGLISH   "set the volume to 40"                              ->  set_volume {"value": 40}
HINGLISH  "volume 40 kar do"                                  ->  set_volume {"value": 40}
ENGLISH   "connect to wifi Redmi Note 12 password hello@123"  ->  connect_wifi {"ssid": "Redmi Note 12", "password": "hello@123"}
HINGLISH  "pwd Tiger@2025 daalo aur Hostel Wing C se jud jao" ->  connect_wifi {"ssid": "Hostel Wing C", "password": "Tiger@2025"}
HINGLISH  "10 min baad shutdown kar dena"                     ->  shutdown {"amount": 10, "unit": "min"}
HINGLISH  "board pin 11 pe led jalao"                         ->  gpio_on {"pin": 11, "numbering": "board"}
HINDI     "5 मिनट का टाइमर लगाओ"                              ->  set_timer {"amount": 5, "unit": "min"}

About the name. "MLM" here means micro language model. It is not a masked language model like BERT: the architecture is a small Transformer encoder-decoder (seq2seq) that writes the command one token at a time.

Executor notice. The model only parses commands; it never runs anything. The executor in this repository is a sample for testing, not a finished product. It gates every action by risk (safe / caution / critical) and asks for confirmation before anything risky. Use it on a test machine and read the command before you run it.

Best use: a personal voice companion for small commands

The model is built to be the "understanding" step of a small, private, always-available assistant: you speak a short command, and your computer or Raspberry Pi does it. It replaces a large LLM for common, well-defined device commands, so the answer is instant (tens of milliseconds), private (nothing leaves the device) and free.

Attach any speech recognizer (ASR) in front of it and an executor behind it:

flowchart LR
    U(["User speaks<br/>'volume 40 kar do'"]) --> A["ASR<br/>speech → text<br/>(Whisper, Vosk, ...)"]
    A -->|"text"| M["<b>tiny-superfast-agentic-MLM-6.5m-v1</b><br/>text → structured command<br/>6.4M params · ~20–70 ms on CPU"]
    M -->|"set_volume {value: 40}<br/>+ confidence"| E["Executor<br/>validate → risk gate → OS command"]
    E --> O(["Output action<br/>volume set to 40%"])
    E -. "clarify / unknown /<br/>low confidence" .-> U

If your viewer does not render Mermaid:

user voice ==> [ ASR ] --text--> [ tiny-superfast-agentic-MLM-6.5m-v1 ] --command JSON--> [ executor ] ==> output action
                                                                                             |
                                          ask the user again  <----  clarify / unknown / low confidence

A good agent pattern: execute only when confidence is high and the risk gate allows it, ask the user when the result is clarify, and hand unknown or low-confidence text to a bigger model or back to the user.

Minimal voice loop (example)

# pip install numpy faster-whisper   (ASR is your choice; this is one option)
import os; os.environ.setdefault("OPENBLAS_NUM_THREADS", "4")      # "2" on a Raspberry Pi 5, see Speed
import sys; sys.path[:0] = ["model", "executor"]
from runtime import CommandModel
import executor as EX
from faster_whisper import WhisperModel

asr = WhisperModel("small", device="cpu", compute_type="int8")
parser = CommandModel("model")

def handle(wav_path):
    text = " ".join(s.text for s in asr.transcribe(wav_path)[0]).strip()   # 1. speech -> text
    out = parser.predict(text); meta = out.pop("_meta")                     # 2. text -> command
    plan = EX.plan(out, None, text)                                          # 3. command -> gated OS command
    if plan.action == "clarify" or meta["confidence"] < 0.8:
        return f"Sorry, can you say that again? ({plan.note or text})"
    ok, why = EX.gate(plan, allow_caution=False)                             # safe actions only in this demo
    return EX.run(plan, out, dry_run=not ok)                                 # 4. act (dry run if gated)

The ASR part of this snippet is an illustration. The model was trained on typed/synthetic text, not on real ASR transcripts, so test it with your own ASR output (see Limitations).

What it understands

Every example is shown in ENGLISH, then the same command in HINGLISH. The outputs, confidences and executor plans are real outputs of this model and the included sample executor.

The last row is a real miss: small talk went to a harmless action, but with low confidence. This is why the agent should check the confidence before acting.

Test results

Accuracy

Held-out evaluation after training (raw model output, before the runtime "snap" repair). "Exact" means the action and every argument match; "Action" means only the action matches.

The same model was also run on the laptop and on the Raspberry Pi 5 (2,340 prompts, with the runtime snap). Both devices produce identical predictions: 97.82% exact, 98.46% action.

Speed and temperature

10 minutes of continuous single-stream inference on each device. Full per-pass and per-minute tables are in eval/BENCHMARK.md.

• Latency grows with the length of the command the model writes: short commands take ~10–35 ms, Wi-Fi commands with an SSID and a password take ~100–250 ms.
• Set the BLAS thread count. The matrices are small, so NumPy's default (one thread per core) is up to 4x slower. Use OPENBLAS_NUM_THREADS=4 on x86 laptops and OPENBLAS_NUM_THREADS=2 on the Raspberry Pi 5 (quickstart.py does this for you).

Test devices

Download and test

1. Get the files

pip install -U huggingface_hub
hf download sraivante/tiny-superfast-agentic-MLM-6.5m-v1 --local-dir tiny-agentic-6.5m
cd tiny-agentic-6.5m

2. Parse commands (nothing is executed)

pip install numpy
python quickstart.py "volume 40 kar do" "connect to wifi Redmi Note 12 password hello@123"
python quickstart.py            # interactive

Each line prints the command, the confidence, the latency, the risk level and the OS command the sample executor would run on this machine.

3. Test UI and checks (Python 3.9+)

The first run creates .venv and installs numpy, fastapi and uvicorn. report really runs the safe, read-only actions on your machine (for example reading the volume or the IP address) and dry-runs everything else.

In the UI, Run all is always a dry run. Execute on a row runs that one command: safe actions run directly, caution actions need the "allow caution" box, and critical actions ask you to type the action name. The UI listens on 127.0.0.1 only and has no login, so do not expose it to a network.

4. Reproduce the speed test

cd eval
sh bench_run_all.sh python ../model <dir with eval .jsonl sets> <out_dir> default 1 2 4

How the model works

• Input: one command, up to 128 characters, normalised and encoded character by character (200-symbol vocabulary), so typos, code-mixing and odd spellings are handled without a word tokenizer.
• Network: Transformer encoder-decoder, d_model 256, 8 heads, FFN 768, 5 encoder + 3 decoder layers (pre-LN).
• Output: a short canonical command such as connect_wifi ssid="…" password="…", written token by token (1,052-token output vocabulary).
• Grammar-constrained decoding: at every step only tokens that keep the output valid are allowed. The action must come from the 371-action catalog, the slots must belong to that action, and text slots (SSIDs, passwords, file names, apps) must be copied from the input. The model cannot invent an action or a password that you did not say.
• Snap: a small post-decode repair extends copied SSIDs and passwords to whole tokens and recovers the original letter case from the input.
• Training: from random initialisation (no pretrained base), 1,230,162 synthetic English/Hinglish rows, 18 epochs on one NVIDIA A100 (bf16, batch 1024, lr 3e-3), about 36 minutes.
• Catalog metadata: each action carries risk (safe/caution/critical), needs_sudo and platforms, so the executor can decide what to run, confirm or refuse.

Safety

• The model does not refuse anything. The executor is the policy: whitelisted command templates, per-shell quoting, a risk gate and clarification questions.
• Credential guard: if a parsed Wi-Fi password or network name looks like a slip (a filler word such as "hai" or "pehle", the password equal to the SSID, the password inside the SSID), the executor asks again instead of connecting.
• Bare pin numbers: "pin no 11" can mean physical pin 11 or GPIO 11. For any action that drives a pin, the executor asks which one unless the user said gpio, bcm, board, physical or header. Power and ground header pins are always refused.
• Critical actions (shutdown, GPIO writes, run_command, ...) need the action name typed as confirmation.

Limitations

• Training data is synthetic, with no real ASR transcripts. Test with your own speech recognizer, and expect lower accuracy on noisy transcripts.
• English and Roman-script Hinglish dominate the training data. Devanagari Hindi is only 0.26% of it (3,232 rows). Spot checks work (क्रोम खोलो → open_app chrome, वाईफाई बंद करो → wifi_off, 5 मिनट का टाइमर लगाओ → set_timer), but there is no Devanagari test set, so its accuracy is not measured.
• One command per utterance. It is a parser, not a multi-step planner.
• Small talk and out-of-scope requests usually come back as unknown, but not always (see kya haal hai bhai above). Use the confidence score.
• Known confusions: scan_lan vs discover_devices (a labelling ambiguity), and bare GPIO pin numbers (handled by the executor, see Safety).
• The sample executor is not a finished product. Some Windows actions have no command-line equivalent and only open the right Settings page. Many Raspberry Pi actions need tools such as pinctrl, nmcli or i2c-tools.

Files

Load the PyTorch checkpoint:

import torch; from pytorch.model import Seq2Seq
ck = torch.load("pytorch/model.pt", map_location="cpu"); c = ck["cfg"]
m = Seq2Seq(c["n_in"], c["n_out"], d=c["d"], h=c["h"], f=c["f"], n_enc=c["n_enc"], n_dec=c["n_dec"],
            max_in=c["max_in"], max_out=c["max_out"])
m.load_state_dict(ck["state"])   # 6,441,472 parameters

License and credits

Code and model weights: Apache-2.0, Copyright (c) 2026 sraivante. Trained from scratch; no pretrained weights are used. The training dataset is synthetic and is not published. Provided as-is, without warranty. You are responsible for anything you let an executor do.