turintech/Qwen3.5-9B-Artemis-GGUF

TurinTech models, built with Artemis

Qwen3.5 9B for local chat and tool calls on a GPU with room for a 5.7 GiB peak.

• 5.661 GB download for the default file, Qwen3.5-9B-Artemis-Q4_K_M.gguf.
• Peak GPU memory 5.7 GiB on an NVIDIA RTX 5090 with an 8,192-token context; the original peaked at 15.9 GiB.
• Its next token matches the original's on 94.0% of WikiText-2 tokens, against 92.9% for Unsloth's Q4_K_M (5.681 GB). That text also chose the file.

Run it

Needs llama.cpp with Qwen3.5 and HTTPS support. -hf also downloads the vision projector and loads it; add --no-mmproj for text only, which skips the 922 MB download.

llama-server -hf turintech/Qwen3.5-9B-Artemis-GGUF:Q4_K_M --port 8080 -c 8192 --jinja

Request example, images, settings and Ollama

The tool-calling results use greedy decoding with thinking off. The quickstart and the Ollama packages do not set that full configuration; set chat_template_kwargs.enable_thinking explicitly.

llama-server -m Qwen3.5-9B-Artemis-Q4_K_M.gguf --port 8080 -c 8192 --jinja

--jinja enables the chat template. In another terminal, wait for curl -s http://localhost:8080/health to return {"status":"ok"}, then send a tool request with thinking off:

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

In our test run (Q4_K_M file downloaded from this repository, CPU, llama.cpp b11617) the reply's tool_calls held get_weather with {"city":"Paris","unit":"celsius"}.

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16.

hf download turintech/Qwen3.5-9B-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m Qwen3.5-9B-Artemis-Q4_K_M.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 16 right (16 with the F16 one), against 14 for Unsloth's Q4_K_M with Unsloth's projector (see Image questions).

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-9b-artemis:q4_k_m

latest selects q4_k_m. The explicit tags below use the same model bytes as the corresponding Hugging Face files.

GB is download size, not runtime memory. Packages use Ollama's Qwen3.5 renderer and tool parser, with temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5. The comparison benchmarks used llama.cpp with different sampling settings; these packages do not establish the same benchmark results in Ollama.

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull.

The packages hold the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/Qwen3.5-9B-Artemis-GGUF:Q4_K_M

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this model brought its projector along and ollama show listed vision. A local package of our Q4_K_M file and mmproj-F16.gguf also listed vision, answered 16 of the 20 chart questions and returned get_weather with Paris and celsius for the tool request above. To build that package, put both files next to a Modelfile with two FROM lines:

FROM ./Qwen3.5-9B-Artemis-Q4_K_M.gguf
FROM ./mmproj-F16.gguf
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER num_ctx 8192

Choose this model

• Best for chat and tool calls; on 258 live tool calls it got 221, the original 216.
• Consider another file when you want the closest match to the original: Unsloth's UD-Q4_K_XL (5.966 GB) is closer on WikiText-2, and our Q5_K_S (6.155 GB) is closer still. No paired task test separated these files.

More on choosing

On the RTX 5090 suite's independent tasks, which no recorded selection run used, no paired test separated any of our three files from the BF16 original or from the size-matched public files after correction. The default Q4_K_M got 238 of 250 GSM8K problems, 147 of 280 MMLU-Pro questions, 221 of 258 BFCL live tool calls and 456 of 541 IFEval prompts right, against 242, 149, 216 and 459 for BF16. On the two texts used in choosing the files, its KL from BF16 was about half that of Unsloth's Q4_K_M (.0145 against .0275 on WikiText-2); Unsloth's larger UD-Q4_K_XL was closer, at .0135. In an earlier GB10 replay its WikiText perplexity ratio to BF16 was higher than Unsloth's Q4_K_M (1.012493 against 1.005114) despite the lower KL. With all layers on an RTX 5090 it peaked at 5.7 GiB of GPU memory.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/Qwen3.5-9B-Artemis-GGUF:Q4_K_M.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/qwen3.5-9b-artemis:<tag>.
• LM Studio's Hub entry turintech/qwen3.5-9b-artemis offers every model file above, without the importance matrix or the vision projector.

The vision projector is optional, for image input. There is no MTP file.

How the files compare

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Results

Our default file against Unsloth's Q4_K_M, the closest public file in size, and the original. Same next token is on WikiText-2, the text the files were chosen on; the other columns are tasks no selection run used.

The files were chosen on closeness over WikiText-2, so that column is not an independent test. On the independent tasks no paired test separated our files from the original or the matched public files. How it was tested.

KL divergence from the BF16 original over 60 chunks of the WikiText-2 test set, measured on an NVIDIA RTX 5090 with llama.cpp b11617; lower is closer, and the original is the reference at zero. The files were chosen on KL over this text, so it is not an independent test, and KL is not task accuracy: on the independent tasks no paired test separated our files from the original or the matched public files. Our IQ4_XS and Q4_K_M have about two thirds and about half the KL of Unsloth's files with the same labels; Unsloth's larger UD-Q4_K_XL and Q5_K_S are closer than our Q4_K_M and Q5_K_S, and in the GB10 replay Unsloth's Q4_K_M had the lower WikiText perplexity ratio despite its higher KL. See How it was tested.

Memory

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated.

Peak memory was measured on an NVIDIA RTX 5090 with llama.cpp b11617, all layers on the GPU, an 8,192-token context, F16 K and V caches and one slot. Each file ran in its own llama-server process with the prompt cache off (--cache-ram 0) and answered the first 25 items of each benchmark task. GPU memory is the peak from nvidia-smi; host memory is the server's peak resident memory, which counts the parts of the memory-mapped model file that were read.

The host figures need --cache-ram 0. The commands on this card keep llama-server's default 8 GiB prompt cache, which lives in host memory: over the full benchmark, host memory grew to 9.3 to 9.9 GiB for every quantized file as the cache filled (9.4 GiB for the default Q4_K_M), while GPU memory stayed the same. Add --cache-ram 0 to the command to stay near the table's host figures. CPU-only memory was not measured. The context length and other runtime settings change these numbers.

With the projector loaded (mmproj-F16.gguf, RTX 5090, llama.cpp b11889, -ngl 99 -c 8192 -np 1 --cache-ram 0), 7.0 GiB of GPU memory was in use after the 20 chart questions.

How it was tested

Measurements, comparisons and limitations

NVIDIA RTX 5090 suite, October 2026

In October 2026 every file below ran on one NVIDIA RTX 5090 (AMD EPYC 7V13 host) with llama.cpp b11617 (14310190c), all layers on the GPU, an 8,192-token context, flash attention on, F16 K and V caches, greedy decoding, thinking off and one slot. Every file was served with Qwen's original chat template and compared with a BF16 conversion of Qwen/Qwen3.5-9B revision c202236, made without the MTP layer for this run. Its SHA-256 differs from the producer conversion used in the GB10 replay further down, for a reason not established, so compare KL within one table. Each file was checked by SHA-256 before the run, our IQ4_XS included, so these are measurements of the downloadable files.

• KL divergence and next-token agreement over 60 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations.
• Native tool calling on BFCL simple (Python), all 400 tasks, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K maths, the first 250 test problems, replies capped at 768 tokens.
• MMLU-Pro, 280 questions (20 from each category), zero-shot with a direct letter answer and replies capped at 8 tokens. This is not the leaderboard protocol, and a reply without a letter counts as wrong.
• IFEval, all 541 prompts, strict prompt-level accuracy with the lm-evaluation-harness v0.4.13 checks, replies capped at 2,048 tokens.

Some of these tasks were used in choosing the published files. The files were chosen on size and on KL over the same two texts, after all 400 BFCL tasks and GSM8K problems 0 to 249 had been run on every candidate. An MMLU-Pro sample of the same size was also run before the choice; its questions were not recorded, so its overlap with this sample is unknown. IFEval was never run on these files before they were published. To test the files on tasks that no recorded selection run used, the run added an independent panel with the same prompts, caps and checkers. These tasks are independent as far as the records go; unrecorded earlier use cannot be ruled out.

• GSM8K fresh: the last 250 test problems (rows 1069 to 1318).
• MMLU-Pro fresh: 280 other questions (20 per category, seed 20261022), none of them in the sample above. The unrecorded earlier sample could overlap with it by chance, about one question in 40.
• BFCL live_simple: all 258 tasks, user-contributed requests with one function to call. No selection record for these files uses this category; 60 of the tasks ran in a check after the files were chosen.
• IFEval, as above.

Before the run, seven comparisons were fixed for this card: each of our files against the original, IQ4_XS against Unsloth's IQ4_XS, Q4_K_M against Unsloth's Q4_K_M, and Q5_K_S against bartowski's Q4_K_M and Unsloth's Q5_K_S, each on the four independent tasks. Only these 28 tests, after a Bonferroni correction for 28, can support a stated difference between two files.

"No tool call at all" counts replies to the 400 BFCL tasks that made no tool call. The texts behind the KL and same-next-token columns, the BFCL tasks and GSM8K were used in choosing the files; IFEval was not.

The paired test compares two files on the same items. It counts the items only one of the two got right and gives the chance of a split at least that uneven if the files were equally good (exact McNemar test). A large p means the test found no difference, which is not proof that the files are equal.

Each cell gives the items only the first file got right, the items only the second got right, and p, uncorrected. After the Bonferroni correction for these 28 tests, none remains, so the card claims no task difference between our files and the original or the matched public files. The smallest p, Q4_K_M against Unsloth's Q4_K_M on live tool calls, is 1.00 after correction.

What the tables show, without a significance claim:

• On KL, our IQ4_XS and Q4_K_M are closer to the original than Unsloth's and bartowski's files with the same labels (.0211 against .0334 and .0229; .0145 against .0275 and .0161 on WikiText-2). Unsloth's UD-Q4_K_XL (5.966 GB) is closer than our Q4_K_M (.0135 against .0145), and Unsloth's Q5_K_S (6.361 GB) closer than our Q5_K_S (.0084 against .0095). Unsloth's Q8_0 is far closer (.0015) and 68% larger than our Q4_K_M.
• bartowski's Q4_K_M got the most MMLU-Pro fresh questions right, 157 against 147 to 149 for ours and 149 for the original.
• Unsloth's Q4_K_M followed 445 IFEval prompts, against 456 for our Q4_K_M and 459 for the original.
• The 768-token cap cut off 12 GSM8K fresh replies from our Q4_K_M against 8 from the original; 4 of the 6 problems only the original got right were cut off in ours.
• Every file called a tool on almost every BFCL task: at most 4 of 400 replies, and 3 to 7 of the 258 live tasks, made no call. Every live_simple task expects a call, so the panel does not test whether a file holds back when no tool fits.
• Almost every MMLU-Pro reply gave a letter: every file gave one on all 280 questions of the earlier sample, and at most one of the 280 fresh replies per file had none.

Fidelity of the downloadable files, 15 October 2026

This replay downloaded and SHA-256 checked each file, including IQ4_XS, and compared it with the original producer BF16 conversion of Qwen/Qwen3.5-9B revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, without MTP (reference SHA-256 0afeac18c93f0814bf30613e6d1b4835fff4d7c69bf1ff31fc12e1486924c6b6). It used NVIDIA GB10, llama.cpp b11617, CUDA, flash attention on, and 60 chunks of 512 tokens each from WikiText-2 test and OASST1 validation. Every row completed both texts. These are reused evaluation texts, not a fresh independent test or proof of historical IQ4_XS tensor identity. The public Unsloth BF16 differs from this producer reference in 69 F32 values; it was not substituted for this replay.

KL, next-token agreement and 99th-percentile KL measure closeness to the original, including its mistakes. Perplexity measures prediction of the observed text; lower KL does not ensure lower perplexity. PPL ratio is llama.cpp’s reported mean PPL(Q)/PPL(base) relative to this BF16 reference, not an accuracy difference. Download sizes include unused MTP tensors where present.

The table reports KL, next-token agreement and perplexity for each file on both texts. The larger Unsloth Q5_K_S has lower chat KL and higher agreement on both texts than our Q5_K_S. Our Q4_K_M has worse WikiText perplexity than Unsloth Q4_K_M. WikiText PPL ratios are 1.011519 for our Q5_K_S and 1.015557 for bartowski Q4_K_M.

The original saved task responses were also rescored offline in this audit; this is not new full-panel inference. Agreement with those saved counts does not resolve the IQ4_XS candidate-to-delivered lineage gap.

Earlier task measurements with verified tensor lineage

Q4_K_M and Q5_K_S retain the tensors of their historical evaluated candidates. The original IQ4_XS candidate-to-delivered linkage remains unresolved, so its historical BFCL, GSM8K and MMLU-Pro counts are omitted. The GB10 fidelity replay measures the delivered IQ4_XS directly, and the RTX 5090 suite at the top of this section measures it on tasks; neither establishes those old task scores.

The earlier task runs used NVIDIA GB10 and llama.cpp b11617, Qwen's original chat template, greedy decoding and thinking off. BFCL v3 simple used the native tools API and bfcl_eval 2026.3.23 AST checker; a response without a parsed tool call counted as wrong. Items 0–199 were development data and 200–399 were held out from that search. Six candidates ran on the latter panel; the published files were chosen on size and KL after those results were available. Treat this as selection-exposed evidence, not independent confirmation. GSM8K used the first 250 test problems; MMLU-Pro used 20 questions per category, 280 total, with letter-only answers. These two tasks used four concurrent requests on a shared GPU; batching can change greedy outputs. The local MMLU-Pro setup is not comparable to Qwen's published results.

Paired exact McNemar tests on the 200 selection-exposed tool tasks (only ours right / only the comparison right): Q4_K_M vs Unsloth Q4_K_M, 2/1 (p=1); Q5_K_S vs bartowski Q4_K_M, 3/0 (p=0.25). Against BF16: Q4_K_M 0/0 (p=1), Q5_K_S 2/0 (p=0.5). These p-values do not account for candidate selection. No task gain or equivalence is established; close totals and nonsignificant tests do not prove equal quality.

Fresh GB10 probe, 15 October 2026

This separate check used NVIDIA GB10, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, from a category that no selection record for these files uses; unrecorded earlier use cannot be ruled out. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

BF16 scored 48/60. In table order, six-document sign-flip p-values are 0.03125, 0.0625, 0.03125, 0.15625. BFCL clustered p-values are 0.6875, 1, 1, 1. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 0.5, 1, 0.5, 1. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

Our IQ4 lost two correct calls to Unsloth IQ4 (47 versus 49); our Q5 tied bartowski Q4 at 49. Our default Q4 still has higher mean KL than the larger Unsloth dynamic Q4. Against plain Q4 its average improves, but the chat-debug document is a loss. Against Unsloth Q5_K_S, mean KL is lower overall but higher on code-panel.

Corrected authored tool diagnostic

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

In the four comparison rows above, exact six-family sign-flip p-values are 1, 1, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

Limits and trade-offs

• Unsloth UD-Q4_K_XL (5.966 GB) has lower KL and higher next-token agreement than our default Q4_K_M. Our Q5_K_S is larger than UD-Q4_K_XL; its download-size-matched comparison is bartowski Q4_K_M.
• Bartowski Q4_K_M contains a 15-tensor MTP block (Q8_0/F32), 258.6 MB, unused by plain decoding. Unsloth's compared files and ours have no MTP block. Matching download size is not matching active-weight budgets.
• The task comparisons covered single-function BFCL calls (simple and live_simple) with thinking off, not irrelevant tools, parallel/nested calls, follow-up turns, thinking mode or long-context quality. Separate smoke checks do not establish general quality for those settings.
• The GB10 and RTX 5090 runs used one llama.cpp release, b11617. No speed claim is made; the memory figures hold for the settings given in Memory.

Public comparison files are pinned to unsloth/Qwen3.5-9B-GGUF revision 3885219b6810b007914f3a7950a8d1b469d598a5 (IQ4_XS, Q4_K_M, UD-Q4_K_XL, Q5_K_S) and bartowski/Qwen_Qwen3.5-9B-GGUF revision 182be2f (Q4_K_M); full file hashes were checked. The RTX 5090 suite added Unsloth's Qwen3.5-9B-Q8_0.gguf and bartowski's Qwen_Qwen3.5-9B-IQ4_XS.gguf from the same revisions, each checked by SHA-256. The Q5_K_S comparator has no historical task counts; its task counts come from the RTX 5090 suite.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

Our file got 2 questions right that Unsloth's missed and Unsloth's none that ours missed (exact McNemar p = 0.5); 20 questions are too few to separate the files. Our two projectors have the same tensors, byte for byte, as Unsloth's mmproj-F16.gguf and mmproj-BF16.gguf (unsloth/Qwen3.5-9B-GGUF revision 3885219); only general.* header fields (names, licence, tags) differ. The projectors were converted from Qwen/Qwen3.5-9B revision c202236 with convert_hf_to_gguf.py --mmproj from llama.cpp b11889.

File-level fidelity summary

These values repeat the RTX 5090 measurements above, on a text used in choosing the files; agreement measures closeness to BF16, not task accuracy.

Conversion and calibration sources

The original weights were converted to BF16 GGUF without MTP and quantized for llama.cpp.

The WikiText evaluation and calibration use the same corpus, with test and train splits respectively. Different splits alone do not establish disjoint text.

Calibration used the first half of WikiText-2 raw train, split at an article boundary near the middle: Salesforce/wikitext revision b08601e0, wikitext-2-raw-v1, CC BY-SA 3.0 / GFDL. The importance matrix used 2,453 chunks of 512 tokens and llama-imatrix defaults, with llama.cpp b11617 (commit 14310190c).

The published importance matrix is Qwen3.5-9B-Artemis-imatrix.gguf. Each model is one standalone GGUF file. The importance-matrix file is calibration data for quantization, not needed to run the model; every model file in this repository was quantized with it. No calibration text is included in any file. Headers name TurinTech in general.quantized_by, the published matrix in quantize.imatrix.file, and the calibration data in quantize.imatrix.dataset. For Q4_K_M and Q5_K_S, full tensor-table and payload checks link the released files to the evaluated candidates; the release differences are header metadata. The historical IQ4_XS candidate-to-release linkage has not been verified, so its earlier task measurements should not be treated as verified measurements of the downloadable file. The GB10 fidelity table and the RTX 5090 suite instead come from direct, hash-verified runs of the delivered file.

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-2026-10-23

One NVIDIA RTX 5090, October 2026.

• Reference: BF16 GGUF conversion of Qwen/Qwen3.5-9B revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, without the MTP layer, made with llama.cpp b11617 (17,920,697,248 bytes, SHA-256 91a808eea0f2131fed6f318732fced8db46f8eadc6b0a6531209e3de435f2b91)
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD, 99th-percentile KLD (llama.cpp's 99.0% KLD), Same top p and PPL ratio (Mean PPL(Q)/PPL(base)) are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -t 16 -ngl 99; the reference logs were written with the same command without --kl-divergence. Taken from the committed scoring script with the chunks, threads and GPU layers recorded for each run
• Hardware: one NVIDIA RTX 5090 (AMD EPYC 7V13 host)
• Settings: 16 threads, all layers on the GPU; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings on the card are for the task runs)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: the files were chosen on size and on KL over the same two texts, so these columns are not independent tests
• Source on this card: the table columns "KL chat", "Same next token, chat", "KL wiki", "Same next token, wiki"
• Comparison files: Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-IQ4_XS.gguf, SHA-256 7e918aeca06c52bcb528ea6b04b4ec957e75ee8c0a73138854c0dfcf371ea429; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q4_K_M.gguf, SHA-256 03b74727a860a56338e042c4420bb3f04b2fec5734175f4cb9fa853daf52b7e8; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-UD-Q4_K_XL.gguf, SHA-256 6f5d30666c2d8ae16a306e616d95341dcf3cc46810df84d7e6f5a7d1e4c1b293; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q5_K_S.gguf, SHA-256 be88613a50ba792bd5a2bc07b74bcd97c9ed9dbac028d27a44c6888e712323c3; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q8_0.gguf, SHA-256 809626574d0cb43d4becfa56169980da2bb448f2299270f7be443cb89d0a6ae4; bartowski: bartowski/Qwen_Qwen3.5-9B-GGUF at 182be2fd6c7b, Qwen_Qwen3.5-9B-IQ4_XS.gguf, SHA-256 7d977cc96c2e08616016d967f232083e354691a8a16f345b26f7d782ee5c9601; bartowski: bartowski/Qwen_Qwen3.5-9B-GGUF at 182be2fd6c7b, Qwen_Qwen3.5-9B-Q4_K_M.gguf, SHA-256 d784ce9eda1a5a7b51e8f705a9e6310844bf4f173654d115823c775fdea56d43

Protocol rtx5090-2026-10-23, text wiki (WikiText-2 test):

Protocol rtx5090-2026-10-23, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Protocol gb10-replay-2026-10-15

NVIDIA GB10, 15 October 2026.

• Reference: the original producer BF16 GGUF conversion of Qwen/Qwen3.5-9B revision c202236235762e1c871ad0ccb60c8ee5ba337b9a, without the MTP layer (17,920,696,800 bytes, SHA-256 0afeac18c93f0814bf30613e6d1b4835fff4d7c69bf1ff31fc12e1486924c6b6); a different conversion from the RTX 5090 table's reference
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD, 99th-percentile KLD (llama.cpp's 99.0% KLD), Same top p and PPL ratio (Mean PPL(Q)/PPL(base)) are llama.cpp's own summary lines
• llama.cpp build: b11617; llama-perplexity SHA-256 7daec014799eae1aefe2b114729a255cc439f4fcb1870ab7b5fa41eed964602c
• Command: llama-perplexity -m <file.gguf> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -ngl 99 -fa on; the reference logs were written with llama-perplexity -m <BF16.gguf> -f <text> --save-all-logits <reference-text.kld> -c 512 --chunks 60 -ngl 99 -fa on
• Hardware: NVIDIA GB10
• Settings: flash attention on, all layers on the GPU; KV cache type not set (llama.cpp default)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: these are reused evaluation texts, not a fresh independent test; this replay does not establish the historical IQ4_XS tensor identity
• Source on this card: the table columns "99th pct KL wiki / chat", "KL wiki / chat", "PPL ratio wiki / chat (lower is better)", "Same top token wiki / chat %"
• Comparison files: Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-IQ4_XS.gguf, SHA-256 7e918aeca06c52bcb528ea6b04b4ec957e75ee8c0a73138854c0dfcf371ea429; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q4_K_M.gguf, SHA-256 03b74727a860a56338e042c4420bb3f04b2fec5734175f4cb9fa853daf52b7e8; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-UD-Q4_K_XL.gguf, SHA-256 6f5d30666c2d8ae16a306e616d95341dcf3cc46810df84d7e6f5a7d1e4c1b293; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q5_K_S.gguf, SHA-256 be88613a50ba792bd5a2bc07b74bcd97c9ed9dbac028d27a44c6888e712323c3; bartowski: bartowski/Qwen_Qwen3.5-9B-GGUF at 182be2fd6c7b, Qwen_Qwen3.5-9B-Q4_K_M.gguf, SHA-256 d784ce9eda1a5a7b51e8f705a9e6310844bf4f173654d115823c775fdea56d43

Protocol gb10-replay-2026-10-15, text wiki (WikiText-2 test):

Protocol gb10-replay-2026-10-15, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Protocol gb10-probe-2026-10-15

Fresh probe, 15 October 2026 (NVIDIA GB10, all layers on the GPU).

• Reference: Unsloth's public BF16 GGUF of Qwen/Qwen3.5-9B, unsloth/Qwen3.5-9B-GGUF at 3885219b6810b007914f3a7950a8d1b469d598a5, file Qwen3.5-9B-BF16.gguf (17,920,697,312 bytes, SHA-256 daebe40eeea7057c1cdf35ac56d13f507d8bf12171bbb7a6b6b0d3f05439159a)
• Text probe6: Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats; chunks: 2 per document; context (tokens): 256; text SHA-256: prose-literature 328e88173fa5ef54cdd72c74ded2968182b4d1e79e89f7dc5ef991c2311740f2; prose-space 872a3d009605ccf22deed05eb9d8eb378d6aad129cd1d26c609f983bdf2a3a53; code-panel 8a86fcd63e889dfffa87a6b5edfe7525f54ea2c2063bb653992e6299e1becd8e; code-runner 41c5af961875d9b5c3e91b82aa3e89c5808408c39293259ee4b402eda07585e1; chat-debug e490eafde70ac0aa94c71bb4b28bc3efaa344d33b4e68becafaf9f022e2eb724; chat-travel 80fbaecbad00bee59eaa569a21d12deea7d6db4635b3e1e887a2e0633ae5102e
• Tool: llama-perplexity --kl-divergence on each of six documents, two chunks of 256 tokens each, against the reference's saved log-probabilities; the value is the plain mean of the six documents' Mean KLD values
• llama.cpp build: b11617 (commit prefix 1431019 from the server fingerprint)
• Command: llama-perplexity -m <file.gguf> --kl-divergence-base <reference-document.kld> --kl-divergence -c 256 --chunks 2 -ngl 99 -fa on for each document; the reference logs were written with llama-perplexity -m <BF16.gguf> -f <document> --save-all-logits <reference-document.kld> -c 256 --chunks 2 -ngl 99 -fa on
• Hardware: NVIDIA GB10, all layers on the GPU
• Settings: llama-perplexity defaults otherwise
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Source on this card: the table column "Mean KL ours / public"
• Comparison files: Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-IQ4_XS.gguf, SHA-256 7e918aeca06c52bcb528ea6b04b4ec957e75ee8c0a73138854c0dfcf371ea429; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q4_K_M.gguf, SHA-256 03b74727a860a56338e042c4420bb3f04b2fec5734175f4cb9fa853daf52b7e8; bartowski: bartowski/Qwen_Qwen3.5-9B-GGUF at 182be2fd6c7b, Qwen_Qwen3.5-9B-Q4_K_M.gguf, SHA-256 d784ce9eda1a5a7b51e8f705a9e6310844bf4f173654d115823c775fdea56d43; Unsloth: unsloth/Qwen3.5-9B-GGUF at 3885219b6810, Qwen3.5-9B-Q5_K_S.gguf, SHA-256 be88613a50ba792bd5a2bc07b74bcd97c9ed9dbac028d27a44c6888e712323c3

Protocol gb10-probe-2026-10-15, text probe6 (Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats):

File names, sizes and SHA-256 checksums

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

Labels are size classes, not a description of every tensor: the smallest standard label whose stock file from the same BF16 has at least as many bits per weight, except where stated below.

Stock llama-quantize dry runs from the same BF16, llama.cpp b11617, give IQ4_XS 4.66, Q4_K_S 4.77, Q4_K_M 5.02, Q5_K_S 5.62 and Q5_K_M 5.77 bits per weight. Our IQ4_XS (4.61) and Q5_K_S (5.49) follow the size-class rule. IQ4_XS is a K-quant mix with no IQ4_XS tensors. Our Q4_K_M (5.05) is an exception: 0.6% above stock Q4_K_M, but smaller than Unsloth's Q4_K_M (5,680,522,464 bytes) and bartowski's (6,169,341,984 bytes). It keeps the Q4_K_M label. Header general.file_type is 30 (IQ4_XS), 15 (Q4_K_M) or 16 (Q5_K_S). Rounded type shares can sum to more than 100%.

File checksums

About and licence

GGUF versions of Qwen3.5 9B for llama.cpp, with the vision encoder as a separate projector file. The multi-token prediction (MTP) layer is not included.

Apache-2.0, as released by Qwen (licence). See LICENSE and NOTICE. This is an independent TurinTech conversion, not affiliated with or endorsed by Alibaba Cloud or the Qwen team.

Anyone serving these files to others is asked to mention TurinTech (turintech.ai) as the source of the quantization. This is a request, not a licence condition.

These files keep improving.