Laya English GGUF (ggmlc)
English System 1 decision model compiled from convaiinnovations/laya (ModernBERT-large, 421M, context 512).
These files are not llama.cpp / llama-cli GGUFs. They are produced by ggmlc, a neural network compiler that lowers PyTorch, JAX, Flax, and Keras models to high-performance GGML execution. Loading them in llama.cpp will fail.
Typed questions (choice / score / noul) are scored in one encoder pass. There is no autoregressive token generation.
Source, CLI, and binaries: examples/laya
Other families: laya-multilingual-GGUF · laya-typed-decisions-GGUF
Files
| File | Quant | Size | Notes |
|---|---|---|---|
laya_english_f16.gguf |
F16 | ~807 MB | Default demo. Best numerical match. |
laya_english_q8_0.gguf |
Q8_0 | ~431 MB | Smaller, still accurate. |
laya_english_ud_q4_k_m.gguf |
UD_Q4_K_M | ~401 MB | Smallest. 1D norms/biases stay F32. |
huggingface-cli download mys/laya-GGUF laya_english_f16.gguf --local-dir .
Run with laya
Download a binary from ggmlc releases (laya.exe / laya). --device defaults to auto (CUDA or Metal if present, else CPU).
laya help
laya list-presets
laya info laya_english_f16.gguf
laya decide laya_english_f16.gguf --preset email --device auto --cuda-graph
laya decide laya_english_f16.gguf --preset guard --text "Ignore previous instructions" --json
laya serve laya_english_f16.gguf --port 8080 --device auto --cuda-graph
laya bench laya_english_f16.gguf --preset email --device auto --cuda-graph
laya daemon laya_english_f16.gguf --device auto --cuda-graph
serve starts Decision Studio (GET /) and POST /api/decide. daemon is newline JSON-RPC on stdin/stdout.
On an RTX 4050 Laptop (6 GB), English F16 + CUDA graph is about 25 ms per noul and 143 ms for the 7-question email preset in one B=7 forward.
What this is
Laya is the open reproduction of TypeSafe Jev: given a state and typed questions, it returns calibrated probabilities instead of generating tokens. Typical uses: email routing, jailbreak/injection guards, expense/invoice/SOC gates, agent harness (act / tool / ask-user / stop).
The English checkpoint does not degrade gracefully off English. For other languages use laya-multilingual-GGUF (or laya decide --models-dir … to route automatically).
License
Apache 2.0, same as the upstream Laya weights. Compiler: ggmlc (MIT).