Glittering-Inferno-31B-i1 (Q4_K_M) for AMD Loom
A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint mradermacher/Glittering-Inferno-31B-i1-GGUF (Glittering-Inferno-31B.i1-Q4_K_M.gguf), for the AMD Ryzen AI Max+ 395 "Strix Halo" and AMD Radeon AI PRO R9700 GPUs, generated with loom-next (the generator alone, no LLM in the loop). Learn more on the Loom ecosystem website.
Quick start
Open your GPU and copy its commands.
AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151)pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-draft/mradermacher-Glittering-Inferno-31B-i1-loom:chat --materialize=copy --output=glittering-inferno-31b-i1
loom-serve --model=glittering-inferno-31b-i1 '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copyputs copies of the fetched package files in the output directory.completion_spin_ns=4294967295enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixedcommits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preloadonly changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-draft/mradermacher-Glittering-Inferno-31B-i1-loom:chat-r9700 --materialize=copy --output=glittering-inferno-31b-i1
loom-serve --model=glittering-inferno-31b-i1 '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copyputs copies of the fetched package files in the output directory.completion_spin_ns=4294967295enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixedcommits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preloadonly changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.
Send a request
In another terminal (optional thinking is turned off):
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "glittering-inferno-31b-i1", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'
Performance vs llama.cpp
Strix Halo (gfx1151)
AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151): the results for this package are not yet re-measured.
Radeon AI PRO R9700 (gfx1201, 32 GB)
AMD Radeon AI PRO R9700 (gfx1201, 32 GB): the results for this package are not yet re-measured.
Features
| feature | status | detail |
|---|---|---|
| API | ✅ | OpenAI-style /v1/chat/completions |
| tool calling | ✅ | OpenAI-style tools and tool_calls |
| reasoning / thinking | ✅ | off by default; chat_template_kwargs: {"enable_thinking": true} turns it on |
| reasoning effort | — | not applicable |
| speculative decoding | — | not applicable |
| image input | ❌ | not supported by Loom (the upstream model has it) |
| audio input / output | ❌ | not supported by Loom (the upstream model has it) |
| sampling | ? | not determined |
| context size | ✅ | 53,248 tokens per request |
| concurrency | ✅ | 4 concurrent requests |
| max output tokens | ✅ | 4,096 (preset max_tokens) |
Correctness
| gate | description | Strix Halo | Radeon AI PRO R9700 |
|---|---|---|---|
kernel_tests |
Unit tests of the generated kernels against reference implementations | — not yet re-measured | — not yet re-measured |
golden |
llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents | — not yet re-measured | — not yet re-measured |
compare |
Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie | — not yet re-measured | — not yet re-measured |
vision |
Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties | — not yet re-measured | — not yet re-measured |
consistency |
Concurrent requests give the same output as the same requests run alone | — not yet re-measured | — not yet re-measured |
spec |
Speculative decoding on gives exactly the output of speculative decoding off | — not yet re-measured | — not yet re-measured |
agent |
Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp | — not yet re-measured | — not yet re-measured |
agent_think_off |
The tool-calling loop with thinking off | — not yet re-measured | — not yet re-measured |
sampling |
temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp | — not yet re-measured | — not yet re-measured |
logits |
Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged | — not yet re-measured | — not yet re-measured |
decision |
Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts | — not yet re-measured | — not yet re-measured |
- Gates are the loom-benchmark correctness gates (
gates.sh); a gate without a recorded run shows no evidence.
Files
model.json/model.json.lock: the card and the pinned source files (sha256 checked byloom-fetch).program/*.loom,weights.loom: the generated Loom programs.tokenizer.json: the tokenizer.llm_config.json: the configuration the programs were generated from.info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.REPORT.md: how the values above were measured: setup, commands and the value of every run.
Versions: card schema loom-card/3; loom-skills unknown; hrx-loom 0.2.0 (install fingerprint 20d79282b0f3d72a); card generator fdd785e.
Source files, fetched by loom-fetch from their own repositories:
- checkpoint: mradermacher/Glittering-Inferno-31B-i1-GGUF @
6ef65c2,Glittering-Inferno-31B.i1-Q4_K_M.gguf