Union Command v10: offline Hinglish/English command parser for laptop and Raspberry Pi agents

Union Command v10 turns one short natural-language command into a structured, typed function call that an agent can execute. It understands English, Hinglish (Roman-script Hindi-English) and a little Devanagari Hindi, and covers 371 actions in 17 categories: audio, display, Wi-Fi, files, apps, browser, timers, services, Docker, local LLMs, Raspberry Pi GPIO/I2C and more.

ENGLISH   "set the volume to 40"          ->  set_volume {"value": 40}
ENGLISH   "set gpio 17 high"              ->  gpio_on {"pin": 17}
ENGLISH   "shut down in 10 minutes"       ->  shutdown {"amount": 10, "unit": "min"}
HINGLISH  "volume 40 kar do"              ->  set_volume {"value": 40}
HINGLISH  "gpio 17 ko high karo"          ->  gpio_on {"pin": 17}
HINGLISH  "10 min baad shutdown kar dena" ->  shutdown {"amount": 10, "unit": "min"}

Executor notice. The model only parses commands. The executor included in this repository is a sample for testing, not a fully developed product. It is there so you can see a parsed command turn into a real action on your own machine. By default it refuses every critical action (shutdown, delete, run_command, GPIO on/off, service changes, ...). Use it on a test machine and read the plan before you run anything.

What it is for

The model is the "understanding" step of an on-device agent: speech-to-text (for example Whisper) → Union Command v10 → validator/risk gate → executor (tool call). It replaces a large LLM for the common, well-defined device commands, so the reply is instant, private and free.

Every use case is shown in ENGLISH first, then the same command in HINGLISH. The outputs are the real predictions of v10.

For an LLM agent, send unknown or low-confidence text on to the bigger model. Vague commands such as turn it off come back as clarify, so the agent can ask the user what to switch off.

Every prediction returns JSON with the action, typed arguments, a confidence score, the top-3 alternative actions and schema validation errors:

{"action": "set_timer", "args": {"amount": 5, "unit": "min"}, "confidence": 0.9998,
  "alternatives": [{"action": "set_timer", "score": 0.9998}, ...], "validation_errors": [], "latency_ms": 6.1}

A good agent pattern: execute only when validation_errors is empty and confidence is high, ask the user when the action is clarify, and hand unknown or low-confidence text to a bigger model.

Categories, sample commands, outputs and executor tools

One intent per category: all ENGLISH commands first, then the same commands in HINGLISH. "Model output" is the real prediction of v10. "Executor tool" is what the sample executor would run for that output on each platform; the tag in brackets shows whether that adapter was ready on the test machine. "Result" is shown only for read-only commands that were actually executed; other rows were planned, not run.

All 72 example commands: 36 intents, each in ENGLISH and HINGLISH

All 72 illustrative commands are in examples/sample_commands.jsonl, with per-platform executor plans in examples/category_examples_windows.json and examples/category_examples_pi.json. v10 got 69 of 72 exactly right (ENGLISH 35 of 36, HINGLISH 34 of 36). The misses:

• ENGLISH run command "ls -la" gave run_command {"command": "command \"ls -la"}; expected run_command {"command": "ls -la"}.
• HINGLISH command "ls -la" chalao gave run_command {"command": "command \"ls -la"}; expected run_command {"command": "ls -la"}.
• HINGLISH ise band kar do gave close_window {}; expected clarify {}.

These commands were written for this demo, so they are illustrative. The accuracy numbers below come from the held-out test set.

Test results

Accuracy

Metric: exact match, meaning the action and every typed argument must be identical to the label. Errors and abstentions stay in the denominator. Same checkpoint on both devices: SHA-256 c91d44e4a687152cd65d86d56cfc0e453a60a1afe14f8c395e8af058f2fc4d5b.

The held-out test split is made of whole template families that were never used for training. It was evaluated once for this release. Practical dev and golden dev guided earlier development, so treat them as regression checks. Earlier development measurements on the same checkpoint (Windows CPU): Grouped dev (reserved template groups) 96.50% (30,596/31,707), Validation 98.22% (23,467/23,892).

Held-out test by language:

Held-out test by category:

Most frequent action confusions on the held-out test:

Speed

Warm, single-command latency over one command per catalog action (371 commands, 2 passes), then batched throughput (batch 32). Includes Python and PyTorch overhead. "RAM" is the whole process after loading the model.

Full accuracy run speed (batched, held-out test):

Temperature

Raspberry Pi 5 notes. The first attempt ran the held-out test in one go with 4 threads. After 202 s and 15,040 commands the SoC reached about 85 °C with the fan near 9,900 RPM, and the Pi dropped off the network until it was power-cycled. The reported Pi results come from a second, gentler run: 2 threads, the held-out test split into five chunks of 5,000 rows, and a cool-down below 65 °C before each chunk. Temperature, the 5 V rail, throttle flags and fan speed were logged every 5 seconds (test_results/rpi5/sensors.log). That run finished with throttle flags at 0x0 throughout, a peak of 73.2 °C and a 5 V rail between 5.05 V and 5.20 V. The Pi made exactly the same predictions as the laptop: the same 1,314 commands were wrong on both devices across the three sets. On the Pi, 1 thread gives the lowest single-command latency; extra threads mainly help batched throughput. For long jobs on a Pi 5, use 1 or 2 threads, keep the active cooler and a 5 V / 5 A supply, and watch vcgencmd measure_temp and vcgencmd get_throttled.

Test devices

Download and test

1. Get the code and the model

git clone https://github.com/sraivante/tiny-agentic-home-robotic-for-edge-device-v10.git
cd tiny-agentic-home-robotic-for-edge-device-v10
python download_model.py      # 96.2 MB from Hugging Face, SHA-256 verified

Or download the ZIP of the repository. The model file alone is at sraivante/tiny-agentic-home-robotic-for-edge-device-v10.

2. Install (Python 3.11 to 3.13, CPU only)

Windows (PowerShell):

python -m venv .venv
.venv\Scripts\activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt

Raspberry Pi OS 64-bit / Linux:

python3 -m venv .venv
. .venv/bin/activate
pip install torch --index-url https://download.pytorch.org/whl/cpu
pip install -r requirements.txt

3. Parse commands (nothing is executed)

python quickstart.py                                  # demo: ENGLISH first, then HINGLISH
python quickstart.py "turn off the wifi" "wifi band karo"
python quickstart.py --json "set a timer for 5 minutes"

From Python:

from tinyagent.runtime import Predictor
model = Predictor("models/a100_minilm_v10_quoted/best.pt", threads=2)
print(model.predict("set the volume to 40"))    # ENGLISH
print(model.predict("volume 40 kar do"))        # HINGLISH

4. Reproduce the tests on your device

python scripts/evaluate.py --data examples/sample_commands.jsonl --output my_eval.json
python scripts/benchmark.py --threads 1,2,4 --output my_benchmark.json

evaluate.py accepts any JSONL file with text, action and args. The full training and test datasets are not published; the measured reports are in test_results/.

5. Try the sample executor lab (optional)

run.bat              # Windows
bash run.sh          # Linux / Raspberry Pi

The launcher installs what it needs into .venv, downloads the model if missing and opens http://127.0.0.1:8770. Pick a category and an example (or type your own), then press Predict only to see the parse, or Run & execute to let the sample executor act on this computer.

• It listens on 127.0.0.1 only and needs a per-session token for every request.
• File actions are confined to runtime/files/.
• critical actions are refused unless you start it with --allow-risk critical.
• Many adapters need extra tools or hardware (pycaw, pywinauto, pinctrl, i2c-tools, NetworkManager, Docker, Ollama). The UI shows which ones are ready. See executor_config.example.json for the integrations that need your own settings.

The executor is a sample for testing, not a finished product. Adapter coverage, error handling and security hardening are incomplete. Do not expose it to a network or run it unattended.

How the model works

• Inputs: one command, up to 256 characters.
• Encoders: a character-level Transformer (width 128, 3 layers) for typos and code-mixing, plus the MiniLM-L6 word encoder from sentence-transformers/all-MiniLM-L6-v2 (fine-tuned), fused per character.
• Heads: a 371-way action classifier and typed argument heads. Each argument is either copied as an exact span of the input (names, paths, passwords, numbers) or chosen from a learned label set (units, modes, apps).
• Decoding v3: keeps quoted file names, host names and whole numbers intact when a span overlaps them.
• Validation: every output is checked against the catalog (union_catalog.json): required slots, types, ranges such as volume 0 to 100 and BCM pin 0 to 27.
• Training: 1,143,182 synthetic fitting rows, NVIDIA A100 (bf16), epoch 5 of 9 selected by grouped-dev accuracy.
• Catalog metadata: each action carries risk (safe/caution/critical), needs_sudo and platforms, so your executor can decide what to run, confirm or refuse. The model never refuses anything itself.

Limitations

• Training data is synthetic. There is no real speech-to-text output in it, so test with your own ASR transcripts.
• Devanagari Hindi coverage is small. English and Roman-script Hinglish dominate.
• One command per utterance. It is a parser, not a multi-step planner.
• Weakest areas on the held-out test: near-duplicate GPIO/I2C actions (i2c_read vs i2c_write, gpio_blink vs gpio_pulse), a few near-synonyms (get_public_ip vs get_ip, scan_lan vs discover_devices) and some free-text argument boundaries, like the run_command example above.
• The model does not refuse dangerous requests. Always put a validator and a risk gate in front of any executor.

Files

License and credits

Code and model weights: Apache-2.0, Copyright (c) 2026 sraivante. The word encoder is initialised from sentence-transformers/all-MiniLM-L6-v2 (Apache-2.0); see provenance/base_models/minilm_l6/. The training dataset is synthetic and is not published. Provided as-is, without warranty. You are responsible for anything you let an executor do.