A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint unsloth/Inkling-2-Mini-GGUF (Inkling-2-Mini-UD-Q4_K_XL.gguf), generated on an AMD Ryzen AI Max+ 395 "Strix Halo" with the loom-skills workflow. Learn more on the Loom ecosystem website.
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/unsloth-Inkling-2-Mini-loom:chat --materialize=copy --output=Inkling-2-Mini
loom-serve --model=Inkling-2-Mini '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copy puts copies of the fetched package files in the output directory.completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixed commits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preload only changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.In another terminal (optional thinking is turned off):
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "Inkling-2-Mini", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'
| metric | llama.cpp | Loom | margin vs best llama.cpp | roofline | Loom % of roofline |
|---|---|---|---|---|---|
| decode tok/s, short prompt | 13.6 | 42.7 | +214 % | 112.7 | 38 % |
| decode tok/s, 2k context | 12.6 | 39.6 | +214 % | 108.5 | 36 % |
| decode tok/s, 32k context | — | 29.6 | — | 95.7 | 31 % |
| prefill tok/s, 1k document | 156.6 | 1179.4 | +653 % | 9583.4 | 12 % |
| prefill tok/s, 2k document | 144.6 | 1214.3 | +740 % | 9283.9 | 13 % |
| prefill tok/s, 8k document | 136.2 | 1281.6 | +841 % | 8991.6 | 14 % |
| prefill tok/s, 16k document | 111.9 | 1226.3 | +996 % | 8642.8 | 14 % |
| prefill tok/s, 32k document | — | 1102.3 | — | 8020.5 | 14 % |
| 4 streams, aggregate decode tok/s | 38.0 | 84.6 | +123 % | 379.1 | 22 % |
| speculative decoding, drafted tokens accepted | — | — | — | n/a | n/a |
Methodology
llamacpp-rocm/b1383; the same GGUF, the same machine and session, the same prompts; Loom served with the active wait. Values are the median of 1–2 sessions (4 streams: median of 1); Loom is timed by the client, llama.cpp by its server.-ngl 99 -cram 0 -c 35024 -kvu -np 4.spec gate's requests.roofline.py (loom-skills unknown commit); decode rows with speculative decoding: verify passes × (1 + acceptance × depth 1) tokens per pass, the drafter's cost not counted, with the acceptance measured in the row's own session: decode 60 %, decode at 2k 60 %, decode at 32k 62 %; 4 streams run without drafting, so its ceiling is single-pass; the 4-stream ceiling of this mixture-of-experts model assumes the four streams route to the same experts (the bound a run may not exceed).| feature | status | detail |
|---|---|---|
| API | ✅ | OpenAI-style /v1/chat/completions |
| tool calling | ✅ | OpenAI-style tools and tool_calls |
| reasoning / thinking | ✅ | on by default; chat_template_kwargs: {"enable_thinking": false} turns it off |
| reasoning effort | ✅ | reasoning_effort: none, minimal, low, medium, high, xhigh, max (default high) |
| speculative decoding | ✅ | DFlash, depth 1, on by default; drafter unsloth/Inkling-2-Mini-GGUF |
| image input | ❌ | not supported by Loom (the upstream model has it) |
| audio input / output | — | not applicable |
| sampling | ✅ | temperature, top_p, top_k, seed |
| context size | ✅ | 1,048,576 tokens per request |
| concurrency | ✅ | 4 concurrent requests |
| max output tokens | ✅ | 4,096 (preset max_tokens) |
| gate | description | result |
|---|---|---|
kernel_tests |
Unit tests of the generated kernels against reference implementations | ❌ no evidence |
golden |
llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents | ✅ pass ¹ |
compare |
Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie | ✅ pass ¹ |
vision |
Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties | — not applicable |
consistency |
Concurrent requests give the same output as the same requests run alone | ✅ pass ¹ |
spec |
Speculative decoding on gives exactly the output of speculative decoding off | ✅ pass ¹ |
agent |
Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp | ✅ pass ¹ |
agent_think_off |
The tool-calling loop with thinking off | ✅ pass ¹ |
sampling |
temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp | ✅ pass ¹ |
logits |
Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged | ❌ no evidence |
decision |
Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts | — not applicable |
gates.sh); a gate without a recorded run shows no evidence.model.json / model.json.lock: the card and the pinned source files (sha256 checked by loom-fetch).program/*.loom, weights.loom: the generated Loom programs.tokenizer.json: the tokenizer.llm_config.json: the configuration the programs were generated from.info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.REPORT.md: how the values above were measured: setup, commands and the value of every run.Versions: card schema loom-card/3; loom-skills unknown; hrx-loom unknown; card generator unknown.
Source files, fetched by loom-fetch from their own repositories:
ea0c79b, Inkling-2-Mini-UD-Q4_K_XL.ggufea0c79b, dspark-Inkling-2-Mini-Q8_0.ggufunsloth/Inkling-2-Mini-GGUF