laya-typed-decisions Core ML adaptive FP16
Native Core ML conversion of convaiinnovations/laya-typed-decisions,
pinned source revision f9ab0b228f0fc0f14d873dbc99038f135c2da1b2. Requires laya-coreml >= 0.5.1,
Apple silicon and macOS 15+. Tested on M3 Max. This is an independent conversion.
import laya_coreml as laya
with laya.load("aac6fef/laya-typed-decisions-coreml-adaptive-fp16") as agent:
result = agent.predict("The customer requests a refund.", {
"refund": {"type": "noul", "instructions": "Does the customer request a refund?"}
})
Install with pip install laya-coreml==0.5.1. Pin this repository's commit SHA
with revision= for reproducibility. Inference requires no PyTorch, MLX or Transformers.
The bundle contains B=1/2/4/16 graphs, dense short sequence buckets, 1024-token capacity and up to 32 options per graph. Logical batch/window behavior follows upstream Laya v0.4.12. Compute precision is float16, running on CPU/GPU; this is not the fixed-shape ANE model.
Validation: 63/63 selected answers match upstream FP32, maximum calibrated
probability error 0.00272663476, 100 stable repeated
calls, and full public-output checks. See validation.json. The model files are
byte-identical to the validated dense export; packaging changes provenance metadata
and adds this card/license. These finite checks do not prove universal model accuracy.
Adaptive graphs load lazily. The runtime's default cache retains at most four
extra execution instances, never answers. This improves repeated-shape latency
at a substantial resident-memory cost; shape_cache_size=0 disables extra instances.
Cold/new shapes can be slower. FP16 is not numerically identical to FP32.
The 0.5.1 FP32 benchmark recorded Core ML process peak RSS of 5.10–8.77 GiB;
that is not a full unified/device memory measure or an FP16 memory estimate.
Runtime and qualification evidence. Original single-graph Hub repositories remain unchanged.