A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint lmstudio-community/Ornith-1.0-35B-GGUF (Ornith-1.0-35B-Q4_K_M.gguf), for the AMD Ryzen AI Max+ 395 "Strix Halo" and AMD Radeon AI PRO R9700 GPUs, generated with the loom-skills workflow. Learn more on the Loom ecosystem website.
Open your GPU and copy its commands.
AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151)pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/lmstudio-community-Ornith-1.0-35B-loom:chat --materialize=copy --output=Ornith-1.0-35B
loom-serve --model=Ornith-1.0-35B '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copy puts copies of the fetched package files in the output directory.completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixed commits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preload only changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/lmstudio-community-Ornith-1.0-35B-loom:chat-r9700 --materialize=copy --output=Ornith-1.0-35B
loom-serve --model=Ornith-1.0-35B '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copy puts copies of the fetched package files in the output directory.completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixed commits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preload only changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.In another terminal (optional thinking is turned off):
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "ornith-ai/Ornith-1.0-35B", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'
| metric | llama.cpp | Loom | margin vs best llama.cpp | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 65.7 | 75.1 | +8 % | 109.3 | 69 % | 69.4 |
| decode tok/s, 2k context | 64.3 | 72.1 | +7 % | 107.1 | 67 % | 67.2 |
| decode tok/s, 32k context | 53.7 | 56.7 | +5 % | 82.1 | 69 % | 54.0 |
| prefill tok/s, 1k document | 1268.2 | 2309.5 | +82 % | 11902.0 | 19 % | 962.2 |
| prefill tok/s, 2k document | 1354.2 | 2538.1 | +87 % | 11708.5 | 22 % | 1071.0 |
| prefill tok/s, 8k document | 1360.3 | 2726.9 | +100 % | 10668.0 | 26 % | 1107.9 |
| prefill tok/s, 16k document | 1247.8 | 2438.6 | +95 % | 9537.8 | 26 % | 1008.2 |
| prefill tok/s, 32k document | 1058.2 | 2111.3 | +100 % | 7870.2 | 27 % | 866.6 |
| 4 streams, aggregate decode tok/s | 139.4 | 173.8 | +25 % | 426.7 | 41 % | 132.3 |
| speculative decoding, drafted tokens accepted | n/a | n/a | n/a | n/a | n/a | n/a |
Methodology
llamacpp-rocm/b1374, llama.cpp Vulkan llamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server.1-stream rows: -ngl 99 -cram 0 -c 65536 -np 1; 4 streams: -ngl 99 -cram 0 -c 65536 -kvu -np 4.1-stream rows: -ngl 99 -cram 0 -c 36864 -np 1; 4 streams: -ngl 99 -cram 0 -c 147456 -np 4.devlab-perf jobs); runners differ between sessions.0.2.0 (hrx-system 0cd370f, recovered); source: DevLab Dispatch run 44088689861.9759acd.roofline.py (loom-skills a1745dc); the 4-stream ceiling of this mixture-of-experts model assumes the four streams route to the same experts (the bound a run may not exceed).| metric | llama.cpp | Loom | margin vs best llama.cpp | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 102.9 | 152.2 | +19 % | 310.3 | 49 % | 127.4 |
| decode tok/s, 2k context | 99.7 | 144.1 | +16 % | 304.0 | 47 % | 124.3 |
| decode tok/s, 32k context | 87.4 | 124.7 | +16 % | 233.0 | 54 % | 107.7 |
| prefill tok/s, 1k document | 3298.1 | 6723.7 | +104 % | 43170.0 | 16 % | 2701.4 |
| prefill tok/s, 2k document | 3689.9 | 6886.2 | +87 % | 42468.2 | 16 % | 3024.3 |
| prefill tok/s, 8k document | 3946.8 | 7393.8 | +87 % | 38693.9 | 19 % | 3170.6 |
| prefill tok/s, 16k document | 3710.9 | 6732.7 | +81 % | 34594.6 | 19 % | 2970.9 |
| prefill tok/s, 32k document | 3319.8 | 5847.1 | +76 % | 28546.0 | 20 % | 2639.5 |
| 4 streams, aggregate decode tok/s | 268.8 | 381.8 | +26 % | 1211.0 | 32 % | 304.2 |
| speculative decoding, drafted tokens accepted | n/a | n/a | n/a | n/a | n/a | n/a |
Methodology
llamacpp-rocm/b1375, llama.cpp Vulkan llamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server.-ngl 99 -cram 0 -c 65536 -np 1.-ngl 99 -cram 0 -c 36864 -np 1.devlab-perf jobs); runners differ between sessions.0.2.0 (hrx-system commit unknown, recovered); source: DevLab Dispatch run 43993245028.9759acd.roofline.py (loom-skills 203cf33); the 4-stream ceiling of this mixture-of-experts model assumes the four streams route to the same experts (the bound a run may not exceed).| feature | status | detail |
|---|---|---|
| API | ✅ | OpenAI-style /v1/chat/completions |
| tool calling | ✅ | OpenAI-style tools and tool_calls |
| reasoning / thinking | ✅ | on by default; chat_template_kwargs: {"enable_thinking": false} turns it off |
| reasoning effort | — | not applicable |
| speculative decoding | — | not applicable |
| image input | ✅ | OpenAI-style image_url parts; projector mmproj-Ornith-1.0-35B-BF16.gguf |
| audio input / output | — | not applicable |
| sampling | ✅ | temperature, top_p, top_k, seed |
| context size | ✅ | 262,144 tokens per request |
| concurrency | ✅ | 4 concurrent requests |
| max output tokens | ✅ | 4,096 (preset max_tokens) |
| gate | description | Strix Halo | Radeon AI PRO R9700 |
|---|---|---|---|
kernel_tests |
Unit tests of the generated kernels against reference implementations | ✅ pass (63/63) ¹ | ✅ pass (63/63) ² |
golden |
llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents | ✅ pass ³ | ✅ pass ⁴ |
compare |
Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie | ✅ pass ³ | ✅ pass ⁴ |
vision |
Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties | ✅ pass ³ | ✅ pass ⁴ |
consistency |
Concurrent requests give the same output as the same requests run alone | ✅ pass ³ | ✅ pass ⁴ |
spec |
Speculative decoding on gives exactly the output of speculative decoding off | — not applicable | — not applicable |
agent |
Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp | ✅ pass ³ | ✅ pass ⁴ |
agent_think_off |
The tool-calling loop with thinking off | ✅ pass ³ | ✅ pass ⁴ |
sampling |
temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp | ✅ pass ³ | ✅ pass ⁴ |
logits |
Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged | ? not re-run on this package (the last pass was on a previous build, sha256 8b13c07; the nightly gates re-run it) |
? not re-run on this package (the last pass was on a previous build, sha256 8b13c07; the nightly gates re-run it) |
decision |
Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts | — not applicable | — not applicable |
0.2.0 (hrx-system 0cd370f, recovered).0.2.0 (hrx-system commit unknown, recovered).0.2.0 (hrx-system 0cd370f, recovered).0.2.0 (hrx-system commit unknown, recovered).gates.sh); a gate without a recorded run shows no evidence.model.json / model.json.lock: the card and the pinned source files (sha256 checked by loom-fetch).program/*.loom, weights.loom: the generated Loom programs.tokenizer.json: the tokenizer.llm_config.json: the configuration the programs were generated from.info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.REPORT.md: how the values above were measured: setup, commands and the value of every run.Versions: card schema loom-card/3; loom-skills unknown; hrx-loom unknown; card generator 926f475.
Source files, fetched by loom-fetch from their own repositories:
a133768, Ornith-1.0-35B-Q4_K_M.ggufa133768, mmproj-Ornith-1.0-35B-BF16.gguf