kev-0.8b ONNX (fp32)
An unofficial ONNX export of Kev-0.8B v1.0 (commit bf75a6a8), a decision model that reads a document plus typed questions (choice, noul, score) and returns calibrated probabilities over each question's options. It generates no text.
- The kev-0.8b LoRA adapter is merged into Qwen/Qwen3.5-0.8B-Base @
dc7cdfe2. - The backbone was built with the onnxruntime-genai 0.20.0 model builder as fp32, for the CPU execution provider.
- The
lm_headand the MTP head are excluded. The graph output islast_hidden_state[batch, seq, 1024]. - The pointer head (kev's
PointerHeadfromhead.pt) is shipped separately ashead.bin+head.json.
Files
| File | Contents |
|---|---|
onnx/model.onnx, onnx/model.onnx.data |
Backbone (3.1 GB). Weights are 64 KiB-aligned so onnxruntime can memory-map them. Keep both names together. |
tokenizer.json, tokenizer_config.json |
The base model's tokenizer, which kev itself uses (kev's load_tokenizer(meta.base)). |
head.bin |
float32 little-endian: q.weight [256, 1024], q.bias [256], k.weight [256, 1024], k.bias [256], back to back. |
head.json |
Shapes and float offsets of those tensors, head_dim 256, hidden_size 1024, temperature 2.3510958125672174. |
REPRODUCE_scripts.txt, requirements.txt |
Every script used to build these files, in run order, with pinned versions. |
The graph uses onnxruntime contrib ops (LinearAttention, CausalConvWithState, GroupQueryAttention, MatMulNBits). It runs on onnxruntime 1.32 (Python) and onnxruntime-node 1.30.0.
How to run it
The backbone is Qwen3.5's hybrid Gated DeltaNet architecture, so kev scores each question as its own causal row: [<state> state… <q> instructions… (<opt> option… </opt>)×K <decide>].
- Delimiter ids:
<|fim_prefix|>=248060 (state),<|fim_middle|>=248061 (q),<|box_start|>=248049 (opt),<|box_end|>=248050 (/opt),<|fim_suffix|>=248062 (decide). - User text: tokenize without special tokens, after rewriting
<|name|>to<¦name¦>. - Options:
noulbecomes the options["no", "yes"](answer = p(yes)).scoregives one option per level in order (answer = Σ i·p_i).choicegives"name: description". - Inputs:
input_ids[1, L]attention_maskall ones [1, L]position_ids[3, 1, L], the row 0..L-1 repeated 3 times- empty past caches:
past_key_values.*[1, 2, 0, 256], andpast.*.conv[1, 6144, 3] /past.*.recurrent[1, 16, 128, 128] all zeros
- Head: with h_d =
last_hidden_stateat<decide>and h_j at each option's</opt>:q = Wq·h_d + bqk_j = Wk·h_j + bkz_j = (k_j·q) / sqrt(256) / temperaturep = softmax(z)
Verification
These are checked against kev's own PyTorch code (kev @ 2074cc4, bf16 on CPU; fp32 did not fit on the 3 GB test machine) on 3 short requests (one choice, one noul, one score):
- Max absolute probability difference 0.0067, same top answer 3/3. Part of that gap is bf16 rounding in the reference.
- onnxruntime-node matches the Python onnxruntime results to within 1e-6.
- Latency is about 4–6.5 s per request on 2 CPU threads of a small laptop.
This is a small smoke test, not a re-run of kev's evaluation suites.
Links and credit
- Kev by Jared Palmer: https://github.com/jaredpalmer/kev, model jaredpalmer/kev-0.8b (Apache-2.0).
- Base model by the Qwen team: Qwen/Qwen3.5-0.8B-Base (Apache-2.0).
- Exported for and used by PromptForge: https://github.com/Codeuctivity/PromptForge.