laya-multilingual-coreml-ane-w8

Laya typed decisions on Apple Silicon, using CPU + Neural Engine. This is a portable Core ML bundle for laya-coreml, converted from convaiinnovations/laya-multilingual. It outputs choice, score, and noul probabilities with zero generated tokens. Inference needs no PyTorch, Transformers, MLX, remote code, or cloud API.

Run

Apple Silicon, macOS 15+, Python 3.11–3.13. Tested on M3 Max / macOS 27.2.

pip install laya-coreml

import laya_coreml as laya

agent = laya.load("aac6fef/laya-multilingual-coreml-ane-w8")  # Download once; Core ML runs locally.
result = agent.predict(
    "The customer asks for a refund of a duplicate payment.",
    {"refund": {"type": "noul", "instructions": "Does the customer request a refund?"}},
)
print(result["answers"])

To download explicitly and then run entirely offline:

hf download aac6fef/laya-multilingual-coreml-ane-w8 --local-dir models/ane-w8
pip install 'laya-coreml[demo]'
laya-coreml-snake --model models/ane-w8 --fps 12

Use laya.load("aac6fef/laya-multilingual-coreml-ane-w8", local_files_only=True) for a cached snapshot or pass a local directory. Use revision="<Hub commit SHA>" to pin a remote revision.

Format and fidelity

This is a separately identified 8-bit grouped K-means weight-palette variant. Activations still use FP16, and the host action head uses FP32. It is approximate: maximum calibrated-probability drift on the 59-question short fixture is 0.014393. The long-context fixture has not been validated for this compressed package.

The fixed L96 package agrees with upstream on 59/59 fitting fixture questions and passes 100 repeated calls. Inputs longer than 96 tokens raise an error instead of being silently shortened to fit.

The exported capacity is 96 total tokens, batch 1, and 32 option slots. Questions/options and state share this budget. The ANE short exports reject over-capacity prompts. Snake uses planner features and a visible optional cycle safety shield; survival is not a claim of unaided game intelligence.

coreml_config.json records shapes, source revisions and per-file SHA256 checksums. validation.json contains the packaging-time validation. Port fidelity on this regression suite does not establish general task accuracy or preserved calibration on arbitrary inputs.

Performance and limits

The multilingual ANE L96 FP16 runtime measured 4.98 / 5.31 ms P50 / P95 for one short question on M3 Max; W8 measured 4.88 / 5.23 ms. Whole-system energy per decision improved 2.78× / 3.19×, respectively, against compiled MLX FP16 in that experiment. Those numbers apply to the named short ANE variants, not every bundle, long contexts, or complete Snake frames. The requested 10× improvement was not achieved.

Measurements and scope · General Core ML benchmarks · Snake demo.

Provenance

• Original checkpoint: convaiinnovations/laya-multilingual at 052592a15d198d9ad47da779604259b10b47b7aa.
• Original weights SHA256: 9d628fd971b700382ac6f65920a86f149777b2e748e0c955fb3b19695aa8f204.
• Upstream implementation: NandhaKishorM/laya, commit 6a5819129eb220570792e417e49723d697efd76f.
• Original models and code are by Convai Innovations and contributors, Apache-2.0.
• Independent conversion; not an official Convai Innovations or Apple release.

See LICENSE and NOTICE. Model quality and task/language limitations originate with Laya; this runtime is an inference port, not a newly trained decision model.