A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint unsloth/Qwen3.8-Flash-Next-GGUF (UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf), generated on an AMD Ryzen AI Max+ 395 "Strix Halo" with the loom-skills workflow. Learn more on the Loom ecosystem website.
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/unsloth-Qwen3.8-Flash-Next-loom:vision-mtp --materialize=copy --output=Qwen3.8-Flash-Next-loom
loom-serve --model=Qwen3.8-Flash-Next-loom '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed
--materialize=copy puts copies of the fetched package files in the output directory.completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixed commits the preset's memory pool up front instead of growing elastic backing.On a stock host (8 MiB memlock) start loom-serve as systemd-run --user --pty --same-dir -p SystemCallFilter=~io_uring_setup -p SystemCallErrorNumber=ENOSYS loom-serve ... (no root; IREE falls back to its POSIX proactor), or raise ulimit -l.
AMD Radeon AI PRO R9700 (gfx1201, 32 GB): this package does not fit its 32 GB (it needs about 118 GB).
In another terminal (optional thinking is turned off):
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "unsloth-Qwen3.8-Flash-Next-loom", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'
| metric | llama.cpp | Loom | margin vs best llama.cpp | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 34.4 | 21.0 | -39 % | 50.4 | 42 % | 34.2 |
| decode tok/s, 2k context | 30.8 | 34.8 | +13 % | 47.4 | 73 % | 29.3 |
| decode tok/s, 32k context | 24.2 | 31.7 | +31 % | 42.5 | 75 % | 24.0 |
| prefill tok/s, 1k document | 417.9 | 1179.1 | +182 % | 4829.0 | 24 % | 327.4 |
| prefill tok/s, 2k document | 442.3 | 1316.4 | +198 % | 4770.8 | 28 % | 352.4 |
| prefill tok/s, 8k document | 451.8 | 617.5 | +37 % | 4651.1 | 13 % | 359.0 |
| prefill tok/s, 16k document | 425.5 | 935.1 | +120 % | 4646.6 | 20 % | 340.2 |
| prefill tok/s, 32k document | 378.7 | 1218.3 | +222 % | 4637.6 | 26 % | 309.5 |
| 4 streams, aggregate decode tok/s | 57.2 | 66.6 | +16 % | 135.1 | 49 % | 55.5 |
| speculative decoding, drafted tokens accepted | — | 82 % | — | n/a | n/a | n/a |
Methodology
llamacpp-rocm/b1371, llama.cpp Vulkan llamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server.1-stream rows: -ngl 99 -cram 0 --spec-type draft-mtp --spec-draft-model hf:unsloth/Qwen3.8-Flash-Next-GGUF/mtp-Qwen3.8-Flash-Next-Q8_0.gguf -ngl 99 -c 65536 -np 1; 4 streams: -ngl 99 -cram 0 -c 65536 -kvu -np 4.1-stream rows: -ngl 99 -cram 0 --spec-type draft-mtp --spec-draft-model hf:unsloth/Qwen3.8-Flash-Next-GGUF/mtp-Qwen3.8-Flash-Next-Q8_0.gguf -ngl 99 -c 36864 -np 1; 4 streams: -ngl 99 -cram 0 -c 147456 -np 4.devlab-perf jobs); runners differ between sessions.0.2.0 (hrx-system 0cd370f, recovered); source: DevLab Dispatch run 44088715568.9759acd.spec gate's requests.roofline.py (loom-skills a1745dc); decode rows with speculative decoding: verify passes × (1 + acceptance × depth 3) tokens per pass, the drafter's cost not counted, with the acceptance measured in the row's own session: decode 50 %, decode at 2k 45 %, decode at 32k 37 %; 4 streams run without drafting, so its ceiling is single-pass; the 4-stream ceiling of this mixture-of-experts model assumes the four streams route to the same experts (the bound a run may not exceed).| feature | status | detail |
|---|---|---|
| API | ✅ | OpenAI-style /v1/chat/completions |
| tool calling | ✅ | OpenAI-style tools and tool_calls |
| reasoning / thinking | ✅ | on by default; chat_template_kwargs: {"enable_thinking": false} turns it off |
| reasoning effort | ✅ | reasoning_effort: low, medium, high, xhigh (default xhigh) |
| speculative decoding | ✅ | MTP, depth 3, on by default; drafter unsloth/Qwen3.8-Flash-Next-GGUF |
| image input | ✅ | OpenAI-style image_url parts; projector mmproj-BF16.gguf |
| audio input / output | — | not applicable |
| sampling | ✅ | temperature, top_p, top_k, seed |
| context size | ✅ | 262,144 tokens per request |
| concurrency | ✅ | 4 concurrent requests |
| max output tokens | ✅ | 4,096 (preset max_tokens) |
| gate | description | result |
|---|---|---|
kernel_tests |
Unit tests of the generated kernels against reference implementations | ? not re-run on this package (the last pass was on a previous build, sha256 baf0749; the nightly gates re-run it) |
golden |
llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents | ✅ pass ¹ |
compare |
Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie | ✅ pass ¹ |
vision |
Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties | ✅ pass ¹ |
consistency |
Concurrent requests give the same output as the same requests run alone | ✅ pass ¹ |
spec |
Speculative decoding on gives exactly the output of speculative decoding off | ✅ pass ¹ |
agent |
Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp | ✅ pass ¹ |
agent_think_off |
The tool-calling loop with thinking off | ✅ pass ¹ |
sampling |
temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp | ✅ pass ¹ |
logits |
Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged | — not applicable |
decision |
Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts | — not applicable |
0.2.0 (hrx-system 0cd370f, recovered).gates.sh); a gate without a recorded run shows no evidence.model.json / model.json.lock: the card and the pinned source files (sha256 checked by loom-fetch).program/*.loom, weights.loom: the generated Loom programs.tokenizer.json: the tokenizer.llm_config.json: the configuration the programs were generated from.info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.REPORT.md: how the values above were measured: setup, commands and the value of every run.Versions: card schema loom-card/3; loom-skills unknown; hrx-loom unknown; card generator 926f475.
Source files, fetched by loom-fetch from their own repositories:
766911a, UD-Q4_K_XL/Qwen3.8-Flash-Next-UD-Q4_K_XL-00001-of-00004.gguf766911a, mtp-Qwen3.8-Flash-Next-Q8_0.gguf766911a, mmproj-BF16.gguf