Sandhi-1.0-8M-onnx
The ONNX build of Sandhi-1.0-8M, a small character-level model that analyses a Malayalam word: sandhi split, morphemes, root lemma, grammatical features, IPA and syllables, and a confidence level. It is the fp32 model, exported for ONNX Runtime, and it is what the in-browser demo runs: nithinmanoj10.github.io/ml-Sandhi.
The weights are the same as the main repository's. Read that card for the training, the evaluation and the limitations; they all apply here.
The three graphs
Three graphs rather than one, so that the word is encoded once and each decoder step runs against a fixed-size cache. The names are the ONNX input and output names.
| Graph | Inputs | Outputs |
|---|---|---|
encoder |
x int64 [1, T]: <FULL> then one id per character |
boundary, syllable [1, T, 2] logits; other [1, 25]; memory [1, T, 256]; then one logits tensor per head, in the order of labels.heads in sandhi_meta.json |
cross |
memory |
cross_k, cross_v [3, 2, 8, T, 32] |
step |
token int64 [2, 1], pos int64 [1], k_cache, v_cache [3, 2, 8, 40, 32], cross_k, cross_v, x |
logits [2, 183], k_cache_out, v_cache_out |
The caller runs the encoder and cross once, then calls step up to 40 times: row 0 decodes the root lemma and row 1 the IPA, each greedily, starting from <bos> with a zeroed cache. Everything after the network (cut thresholds, the word-list step, alternatives) is ordinary code. The reference implementation is JavaScript, in ml-Sandhi (docs/js/), with the Python equivalent in ml_sandhi/.
How it was checked
On the 1,007 words of the int8 build's golden set that the pipeline analyses (the other 10 abstain):
- ONNX Runtime against the PyTorch fp32 model: the final analysis is identical on every field (split, features, IPA, syllables, confidence, alternatives, display, morphemes, lemma, root) for all 1,007 words.
- Against the golden answers themselves (made by the int8 build): split, IPA, syllables, confidence, display and morphemes match 100%; features and alternatives 99.8%; lemma and root 99.9%. That is the int8 rounding, seen from the other side.
- The JavaScript pipeline in ONNX Runtime Web is identical to the Python package on every field, for all 1,007 words.
Speed
Measured on a desktop CPU, single-threaded WebAssembly: about 100 ms per word at the median in Node, and about 120–210 ms in Chrome after a first analysis of about 0.5 s. Phones were not measured. A dictionary lookup is faster; the point is coverage.
Limitations
Everything in the main repository's limitations applies: weak on long words with many endings and on real compounds, and this is a pre-freeze model with one of two sanctioned holdout evaluations used. This build has not been evaluated on the holdouts separately; its accuracy is the main repository's, since the answers are identical. It is not quantized.
License
Apache-2.0. Training sources are attributed in NOTICE.