turintech/Qwen3.5-0.8B-Artemis-GGUF

TurinTech models, built with Artemis

A tiny Qwen3.5 0.8B for local tool calls and short chat answers, with all layers on a small GPU.

• 0.569 GB download for the default file, Qwen3.5-0.8B-Artemis-Q5_K_M.gguf.
• Peak GPU memory 1.2 GiB on an NVIDIA RTX 5090 with an 8,192-token context; the original peaked at 2.2 GiB.
• 163 of 258 held-out BFCL live tool calls right, against 109 for bartowski's Q4_K_M (0.580 GB).

Run it

Needs llama.cpp with Qwen3.5 and HTTPS support. -hf also downloads the vision projector and loads it; add --no-mmproj for text only, which skips the 207 MB download.

llama-server -hf turintech/Qwen3.5-0.8B-Artemis-GGUF:Q5_K_M --port 8080 -c 8192 --jinja

Request example, images, settings and Ollama

Keep the :Q5_K_M tag: untagged llama.cpp downloads and Hugging Face's generated Ollama import select Q4_K_M instead.

llama-server -m Qwen3.5-0.8B-Artemis-Q5_K_M.gguf --port 8080 -c 8192 --jinja

-c 8192 sets an 8,192-token context; --jinja uses the chat template stored in the file, which is Qwen's original. The raw template defaults to thinking off when enable_thinking is absent. llama-server can supply its own default; in our b11617 runtime check, an omitted flag enabled thinking. In our test, a short question with thinking on used all 1,500 output tokens thinking and returned no answer, so turn thinking off in the request for short answers and tool calls, as below.

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

In our test run the reply's tool_calls held get_weather with {"city":"Paris","unit":"celsius"}.

# the smaller file: download it, then the same command
hf download turintech/Qwen3.5-0.8B-Artemis-GGUF Qwen3.5-0.8B-Artemis-Q4_K_M.gguf --local-dir .
llama-server -m Qwen3.5-0.8B-Artemis-Q4_K_M.gguf --port 8080 -c 8192 --jinja

Questions, problems and requests for other small models are welcome in our feedback thread.

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16.

hf download turintech/Qwen3.5-0.8B-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m Qwen3.5-0.8B-Artemis-Q5_K_M.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 11 right (11 with the F16 one), against 11 for Unsloth's Q5_K_M with Unsloth's projector (see Image questions).

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-0.8b-artemis:q5_k_m

latest and q5_k_m are the Q5_K_M file and q4_k_m is the Q4_K_M file, with the same SHA-256 as this repository. The packages use Ollama's Qwen3.5 renderer and tool-call parser, as Ollama's own qwen3.5 package does, and Qwen's recommended sampling for text in thinking mode (temperature 1, top_k 20, top_p 0.95, presence penalty 1.5). To import a file directly with the same settings:

hf download turintech/Qwen3.5-0.8B-Artemis-GGUF Qwen3.5-0.8B-Artemis-Q5_K_M.gguf --local-dir .
printf 'FROM ./Qwen3.5-0.8B-Artemis-Q5_K_M.gguf\nRENDERER qwen3.5\nPARSER qwen3.5\nPARAMETER temperature 1\nPARAMETER top_k 20\nPARAMETER top_p 0.95\nPARAMETER presence_penalty 1.5\n' > Modelfile
ollama create qwen3.5-0.8b-artemis -f Modelfile
ollama run qwen3.5-0.8b-artemis

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull. The accuracy comparisons use llama.cpp with thinking explicitly off.

The packages hold the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/Qwen3.5-0.8B-Artemis-GGUF:Q5_K_M

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this model brought its projector along and ollama show listed vision. A local package of our Q5_K_M file and mmproj-F16.gguf also listed vision, answered 11 of the 20 chart questions and returned get_weather with Paris and celsius for the tool request above. To build that package, put both files next to a Modelfile with two FROM lines:

FROM ./Qwen3.5-0.8B-Artemis-Q5_K_M.gguf
FROM ./mmproj-F16.gguf
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER num_ctx 8192

Choose this model

• Best for tool calls and short chat answers on a small GPU.
• Consider another file when knowledge questions matter: on MMLU-Pro it got 33 of 280 right against 44 for the original, and Unsloth's Q8_0 (0.812 GB) stays far closer to the original.

More on choosing

On the RTX 5090 GSM8K sample (problems 0 to 249, a guardrail in choosing the Q4_K_M), Q5_K_M answered 132/250 correctly and Q4_K_M 137/250, versus 145/250 for BF16. On IFEval (all 541 prompts, strict), which played no part in choosing the files, Q5_K_M followed 308 and Q4_K_M 296, versus 290 for BF16; no paired test separated either file from BF16. Over all 400 tool-calling tasks, every one of which was used in choosing the files, Q5_K_M got 279 right against 292 for BF16 and 290 and 282 for the public Q5_K_M files, while Q4_K_M got 313.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/Qwen3.5-0.8B-Artemis-GGUF:Q5_K_M.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/qwen3.5-0.8b-artemis:<tag>.
• LM Studio's Hub entry turintech/qwen3.5-0.8b-artemis offers every model file above, without the importance matrix or the vision projector.

Without a tag, -hf and hf.co fetch Qwen3.5-0.8B-Artemis-Q4_K_M.gguf, not the default file, so always add the tag.

The vision projector is optional, for image input. There is no MTP file.

How the files compare

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Results

Our default file against bartowski's Q4_K_M, the closest public file in size, and the original. These tasks played no part in choosing the files.

Only the tool-call gap over bartowski's file survives correction. MMLU-Pro is the weak spot: 33 of 280 against 44 for the original. How it was tested.

Correct tool calls on the 258 BFCL live_simple tasks, an independent panel that no recorded selection run used, measured on an NVIDIA RTX 5090 with llama.cpp b11617 and drawn on the full 0 to 258 scale. Our Q4_K_M got the most right, 173 against 153 for the original and 115 for Unsloth's Q4_K_M, and our Q5_K_M 163 against 109 for bartowski's Q4_K_M; these three differences survive correction. The files with fewer right made no tool call more often. No other pre-registered test on the panel separated our files from the original or the matched public files after correction; among those, the largest gap against ours was MMLU-Pro, where the Q5_K_M got 33 of 280 right against 44 for the original. Every task expects a call, so the chart does not show whether a file holds back when no tool fits. See How it was tested.

Memory

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated. The benchmark comparisons used F16 K and V caches.

Peak memory was measured on an NVIDIA RTX 5090 with llama.cpp b11617, all layers on the GPU, an 8,192-token context, F16 K and V caches and one slot. Each file ran in its own llama-server process with the prompt cache off (--cache-ram 0) and answered the first 25 items of each benchmark task. GPU memory is the peak from nvidia-smi; host memory is the server's peak resident memory, which counts the parts of the memory-mapped model file that were read.

A smaller file does not always need less memory: on the RTX 5090 our Q4_K_M peaked higher on the GPU than our Q5_K_M. The commands on this card keep llama-server's default 8 GiB prompt cache, which lives in host memory: over the full benchmark, host memory grew to 9.0 to 9.6 GiB for every file as the cache filled, while GPU memory stayed the same. Add --cache-ram 0 to the command to stay near the table's host figures. CPU-only memory was not measured. The context length and other runtime settings change these numbers.

With the projector loaded (mmproj-F16.gguf, RTX 5090, llama.cpp b11889, -ngl 99 -c 8192 -np 1 --cache-ram 0), 1.7 GiB of GPU memory was in use after the 20 chart questions.

On an NVIDIA RTX 3060, with the table's server settings, no projector and a much shorter workload (about 270 prompt tokens and 128 output tokens per request), peak GPU memory was 824 MiB for our Q5_K_M, 804 MiB for our Q4_K_M and 1,744 MiB for the original; see Speed. Use the table above to size a machine.

Speed

Measured on 2 November 2026 on one desktop with an NVIDIA GeForce RTX 3060 12 GB and an AMD Ryzen 9 5900X, using llama.cpp b11617, the build of the RTX 5090 and M3 Max tables. Prefill is reading a 512-token prompt; generation is writing 128 tokens, in a fresh context or after 4,096 tokens. This measures speed and memory only, not quality; compare rows within one table.

On the GPU, every layer on the RTX 3060:

Time to first token and time between tokens: prompts of 256 tokens, 128 output tokens requested, thinking off, one request at a time.

On the CPU only, 12 threads on the Ryzen 9 5900X:

Against the original, our files generated 1.9 to 2.1 times as fast on the GPU and, on the CPU, 2.2 to 2.4 times as fast in a fresh context and 2.0 to 2.1 times after 4,096 tokens. The time between tokens on the GPU roughly halved and peak GPU memory fell by about half. The public quantized files gained most of this too: in a fresh context, 1.7 to 1.8 times on the GPU and 2.2 to 2.4 times on the CPU. Unsloth's Q4_K_M had the lowest peak GPU memory (788 MiB).

In a fresh context, on this GPU our Q4_K_M generated 15% faster than Unsloth's Q4_K_M of the same size, and our Q5_K_M 16% faster than Unsloth's Q5_K_M. The CPU shows no such lead: our Q4_K_M and Unsloth's Q4_K_M were level (50.7 and 50.6 tokens per second), our Q5_K_M and Unsloth's Q5_K_M were level (46.7 and 46.8), and our Q5_K_M was behind bartowski's Q4_K_M (48.4), Unsloth's UD-Q4_K_XL (49.2) and Unsloth's Q4_K_M (50.6). Prefill differences between the quantized files are within about one standard deviation. This is one GPU, one CPU and one llama.cpp build. An earlier run on a shared RTX 5090 host, too noisy to publish, did not show this GPU lead: Unsloth's Q4_K_M was ahead of our Q4_K_M there. Do not assume the lead on other GPUs.

Speed test settings

• Builds: llama.cpp b11617 (14310190c), built with the CUDA 13.2 toolkit (driver 595.71.05) for the GPU rows and without CUDA for the CPU rows, 12 threads.
• llama-bench: -p 512 -n 128 -fa 1 -r 10, then -d 4096 -r 5 for generation after a 4,096-token context, with -ngl 99 on the GPU and -ngl 0 on the CPU build. Two passes, the second in reverse file order; each value is the mean over both (20 samples, 10 after 4,096 tokens). Prefill shows the standard deviation; for generation it was 2.3 tokens per second or less on the GPU and 1.0 or less on the CPU. The two passes' fresh-context generation means agreed within 2.4% for every file; CPU prefill differed by up to 9.5% between passes (the original).
• Time to first token and between tokens: llama-server -c 8192 -np 1 -ngl 99 --jinja -fa on --cache-ram 0 --chat-template-kwargs '{"enable_thinking":false}', driven by AIPerf 0.13.0 with 30 synthetic prompts of 256 tokens (268 to 270 with the chat template), greedy, prompt cache off, 128 output tokens requested (24 of the 420 measured replies ended earlier, none of them from our files), one request at a time after two warm-ups, in two passes. Medians are over the 60 measured requests (nearest rank); time between tokens is each request's mean inter-token latency. No request failed.
• Peak GPU memory: the server's own row in nvidia-smi, sampled every 2 s, the higher of the two passes.
• No other benchmark or compute job ran. Two unrelated containers on the machine restarted about once a minute during the run.
• Public files: the four public files in the tables, at the revisions listed under What is not better, each checked by SHA-256 on the measuring machine.

How it was tested

Measurements, comparisons and limitations

Every file, ours and public, was compared with a BF16 conversion of the same original (Qwen/Qwen3.5-0.8B revision 2fc0636, converted without the MTP layer):

• KL divergence and next-token agreement over 60 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations.
• Native tool calling on BFCL simple (Python): 400 tasks, each a user request and one function to call. The first 200 were used to choose between candidate files and the other 200 were reserved initially from that search. Each request went through llama-server's OpenAI-style tools API with one slot, greedy decoding, thinking off and Qwen's original chat template for every file, and was scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K maths, 250 problems (the first 250 of the test set on the RTX 5090).
• MMLU-Pro, 280 questions (20 from each category), zero-shot with a direct letter answer and replies capped at 8 tokens. This is not the leaderboard protocol. A reply without a letter counts as wrong, and even the original gave no letter on 117 of 280, so these counts mix answer format with knowledge.
• IFEval, all 541 prompts, strict prompt-level accuracy with the lm-evaluation-harness v0.4.13 checks.

The two KL texts were used in choosing both files, so the KL and same-next-token columns are not independent tests. MMLU-Pro and IFEval played no part in choosing the files. GSM8K problems 0 to 249 served as a pass/fail guardrail in the search that produced the Q4_K_M; GSM8K played no part in choosing the Q5_K_M. In October 2026 the RTX 5090 suite was rerun on a second RTX 5090 machine with the same protocol and llama.cpp build. The rerun used its own implementation of that protocol. It reproduced every earlier value in the RTX 5090 tables at the precision shown, and added the count over all 400 tool-calling tasks, MMLU-Pro, IFEval and five more public files.

The selection-exposed 200 are not fresh. On the Apple M3 Max they helped decide which files to publish. On the RTX 5090 they were run on four final candidates for the Q4_K_M file, and the file here is the one with the most tool calls right among them, so its count may be higher than it would be on new tasks. The October rerun reproduced every selection-panel count.

The paired test compares two files on the same tasks. It counts the tasks only one of the two got right and gives the chance of a split at least that uneven if the files were equally good (exact McNemar test). A large p means the test found no difference, which is not proof that the files are equal.

NVIDIA RTX 5090, llama.cpp b11617 (CUDA)

"No tool call at all" counts replies on the selection-exposed 200 that made no tool call.

Comparisons with the original and with files added in October, on each half of the 400 tool-calling tasks and on all of them. Each cell gives the tasks only the first file got right, the tasks only the second got right, and p, uncorrected.

The same paired test on GSM8K, MMLU-Pro and IFEval. MMLU-Pro and IFEval played no part in choosing the files; GSM8K served as the guardrail described above for the Q4_K_M. Each cell gives the items only the first file got right, the items only the second got right, and p.

The October run made 64 paired tests over the four tasks, with tool calls counted on all 400 tasks, every one of which was used in choosing our files. After a Bonferroni correction for those 64, the differences that remain are on tool calls, plus MMLU-Pro against bartowski's IQ4_XS, which gave no letter on 254 of its 280 replies. On the 400 tool-calling tasks, our two files got more right than the plain Q4_K_M, UD-Q4_K_XL and IQ4_XS files, our Q4_K_M also got more right than bartowski's Q5_K_M, and Unsloth's Q8_0 got more right than our Q5_K_M (36 against 8, mostly in the first 200). No GSM8K or IFEval difference survives the correction, so the card does not claim an IFEval gain. The selection-panel rows and the per-half columns above are outside those 64 tests; every p in those tables is shown uncorrected.

The table reports selection-panel correct-call counts for Q4_K_M and the original. The paired test does not separate the two (p = 0.06), so the card does not claim that it beats the original. Tool-calling counts also move a lot between task sets: on the first 200 tasks the Q5_K_M file got 119 right against 136 for the original, with 58 replies that made no tool call, while on the selection-exposed 200 the paired test did not separate it from the original. The first 200 were the search set for the Q4_K_M file, so its 149 there is not a fair comparison either. The original RTX 5090 rows came from one harness on GPUs of the same model; re-run on a second GPU, the original and the Q4_K_M file scored the same on every selection-exposed task, and the October rerun reproduced all of those rows.

Independent panel, RTX 5090, October 2026

A later run on the machine used for the October rerun, with the same protocol and llama.cpp build (14310190c, b11617), measured the ten files above on tasks that no recorded selection run used. They are independent as far as the records go; unrecorded earlier use cannot be ruled out. It used the same prompts, reply caps and checkers as the tasks above:

• GSM8K fresh: the last 250 test problems (rows 1069 to 1318). The searches behind our files used problems 0 to 255 at most.
• MMLU-Pro fresh: 280 other questions (20 per category, seed 20261022), none of them in the sample above.
• BFCL live_simple: all 258 tasks, user-contributed requests with one function to call. No selection record for these files uses this category; 60 of the tasks ran in the fresh CPU probe below, after the files were chosen.
• IFEval: the 541 prompts above, from the October rerun.

Before the run, six comparisons were fixed for this card: each of our files against the original, Q4_K_M against Unsloth's Q4_K_M, and Q5_K_M against bartowski's Q4_K_M, Unsloth's Q5_K_M and bartowski's Q5_K_M, each on the four tasks. Only these 24 tests, after a Bonferroni correction for 24, can support a stated difference between two files.

Here "No tool call at all" counts replies to the 258 live tasks that made no tool call. The 768-token cap cut off 45 of the original's 250 GSM8K fresh replies, against 31 for our Q5_K_M and 32 for our Q4_K_M.

Each cell gives the items only the first file got right, the items only the second got right, and p, uncorrected. After the Bonferroni correction for these 24 tests, three differences remain, all on live tool calls: our Q4_K_M against the original (25 / 5, corrected p=0.008) and against Unsloth's Q4_K_M (65 / 7, corrected p<0.001), and our Q5_K_M against bartowski's Q4_K_M (63 / 9, corrected p<0.001). The files with fewer right made no tool call more often: 40 times for the original, 95 for Unsloth's Q4_K_M and 109 for bartowski's, against 11 and 17 for ours. Every live_simple task expects a call, so the panel does not test whether a file holds back when no tool fits. No other test survives. The Q5_K_M answered fewer MMLU-Pro fresh questions than the original (2 / 13, 0.18 corrected), and at this size many MMLU-Pro replies gave no letter: 113 of 280 for the original, 137 for our Q5_K_M and 114 for our Q4_K_M.

Apple M3 Max, llama.cpp b11617 (Metal)

The Q4_K_M file in this repository was chosen after the M3 Max run and has not been run on that machine.

Compare rows within one table. The two machines used separate harnesses for the same tests, so the prompts and the chat text sample may not match exactly. The M3 Max rows for our Q5_K_M are a separate build made on the Mac, with the same tensor types and its own importance matrix. It is not the Q5_K_M file published here, which was built on the RTX 5090 machine. The two machines give different absolute counts: the BF16 original got 142 selection-exposed tasks right on the M3 Max and 156 on the RTX 5090, and every file's KL is 6 to 14% higher on the RTX 5090. Both machines' tables include our Q5_K_M, bartowski's Q4_K_M and the original, with paired tests between them. On the selection-exposed 200 neither machine separated Q5_K_M from the original; on the RTX 5090's first 200 the original got 18 tasks right that Q5_K_M missed, against 1. The RTX 5090 runs scored tool calls with the answer key in bfcl_eval 2026.3.23 and the M3 Max run with the BFCL v3 key; the two keys differ on one selection-exposed task.

Most of the tool-calling losses in the public 4-bit files are replies with no tool call at all. On the RTX 5090 selection-exposed 200 these were 89 for bartowski's Q4_K_M, 95 for Unsloth's Q4_K_M, 55 for UD-Q4_K_XL and 147 for IQ4_XS, against 22 for the original.

Fresh CPU probe, 15 October 2026

This separate check used workstation CPU, eight inference threads, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, from a category that no selection record for these files uses; unrecorded earlier use cannot be ruled out. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

BF16 scored 34/60. In table order, six-document sign-flip p-values are 0.03125, 0.03125, 0.03125, 0.03125. BFCL clustered p-values are 0.019287, 0.011353, 1, 0.34375. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 0.5, 0.5, 0.5, 0.5. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

Lower mean KL does not mean every token is closer: on prose-literature, our Q4 maximum token KL is 2.481113 versus Unsloth Q4 1.248548. The advantage over plain Q4 files does not extend to Unsloth’s dynamic Q4 on this tool slice: both scored 39/60. The similarly sized Unsloth Q5_K_S scored 43/60 against our Q5 at 39/60; the lower mean KL of our Q5 did not translate into a higher tool score.

Corrected authored tool diagnostic

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

In the four comparison rows above, exact six-family sign-flip p-values are 0.25, 0.125, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

Separate CPU replay of the original panels

The released files were rerun on the workstation CPU with llama.cpp b11617, eight inference threads, one 8,192-token slot, greedy decoding, seed 20261015 and thinking off. BFCL uses the original 400 simple-Python tasks and official AST checker, with the development and selection-exposed halves kept separate. GSM8K uses the exact original first 250 questions, a 768-token output limit and the original boxed-answer/last-number parser. The public Unsloth BF16 file is the reference; the historical producer BF16 identity is unestablished. These results do not recreate the separate unpublished Mac Q5 build or establish CPU/GPU equivalence.

Boxed-only and nontruncated sensitivities keep all 250 questions in the denominator and treat rejected outputs as failures. They were specified after inspecting truncation and parser behavior. The historical RTX BF16 result is 145/250; this CPU/public-reference result is 143/250, with 35 output-limit hits. Counts can change across artifact, hardware and runtime settings.

These post-hoc paired tests concern this fixed panel and scorer, not general maths quality.

CPU BFCL paired tests

The correction covers all 15 file pairs on both panels. Neither panel is new independent release-selection evidence.

The original WikiText/OASST texts were independently reconstructed to matching hashes and replayed over 60 chunks of 512 tokens each. These are new CPU measurements with the pinned public BF16 reference, separately from the historical hardware/build tables above.

Lower mean KL is not a claim that every token, task or tail metric improves.

What is not better

• Download size is not the same as active-weight size. The pinned bartowski Q4_K_M contains a 15-tensor MTP block with 21,747,712 payload bytes, unused by ordinary decoding. Subtracting that payload from its 579,615,840-byte download gives 557,868,128 bytes, below our Q5_K_M's 569,082,048 bytes. Our files and the tested Unsloth files contain no MTP block. The comparisons above match download sizes, not active-weight budgets.
• Maths: in the historical GSM8K runs every quantized file, ours and public, scored below the original. On the RTX 5090 the original got 145 of 250, our Q4_K_M 137 and Q5_K_M 132, bartowski's Q4_K_M 138 and Unsloth's 131. On the M3 Max the original got 134, our Q5_K_M Mac build 122, bartowski's 129 and Unsloth's 124. Those historical runs supplied no paired GSM8K test. In the October RTX 5090 rerun, Unsloth's Q8_0 (0.812 GB) got 147, the only file above the original, and the public Q5_K_M files got 136 and 137; the paired tests are in the RTX 5090 section. The separate CPU replay above includes paired tests and scoring sensitivities; it does not establish a general maths gain.
• MMLU-Pro: on the M3 Max every file, including the original, got 32 to 39 of 280, and on the RTX 5090 25 to 38 (bartowski's IQ4_XS 8, because it gave no letter on 254 of 280 replies), against about 28 expected from guessing among 10 choices. At this size the test says little about the files.
• Same-label Q5 files: Unsloth's Q5_K_M (0.590 GB) and bartowski's Q5_K_M (0.646 GB, 13.5% larger) are close to our Q5_K_M on KL (.0120 and .0101 against .0098 on WikiText-2) and within a few items of it on GSM8K and MMLU-Pro. On IFEval ours followed 308 prompts against their 295 and 288; no difference on those three tasks survives correction. On tool calls both got more right over the 400 tasks (290 and 282 against 279); on the first 200, Unsloth's got 25 right that ours missed, and ours 4 that it missed. Unsloth's Q8_0 is far closer to the original (KL .0012) and 43% larger than our Q5_K_M.
• Unsloth's UD-Q4_K_XL (0.559 GB) holds up much better on tool calling than the plain Q4_K_M files (104 of 200 selection-exposed on the M3 Max). Our unpublished file at that size recorded KL .0121 against .0215 and tool-calling counts of 112 against 104 (p = 0.23). The tool-calling difference was not significant, so we do not publish it. That decision used the selection-exposed run above.
• The historical tool benchmark covered single-function BFCL simple calls with thinking off. The fresh authored tool and runtime probes have narrower scope than a general task benchmark.
• Speed: on the CPU our files had no lead over the public files, and bartowski's Q4_K_M, Unsloth's UD-Q4_K_XL and Unsloth's Q4_K_M generated 3.5%, 5.2% and 8.1% faster than our Q5_K_M, the default file, in a fresh context, and 1.4%, 2.0% and 5.0% faster after 4,096 tokens. On the RTX 3060, Unsloth's Q4_K_M had the lowest peak GPU memory. The GPU speed lead in Speed was measured on one RTX 3060.
• The RTX 5090 and M3 Max tables use one llama.cpp release (b11617).

Public files: unsloth/Qwen3.5-0.8B-GGUF revision 6ab4614 (Qwen3.5-0.8B-Q4_K_M.gguf; the October rerun added Qwen3.5-0.8B-UD-Q4_K_XL.gguf, Qwen3.5-0.8B-Q5_K_M.gguf and Qwen3.5-0.8B-Q8_0.gguf) and bartowski/Qwen_Qwen3.5-0.8B-GGUF revision f36b1ea (Qwen_Qwen3.5-0.8B-Q4_K_M.gguf; the rerun added Qwen_Qwen3.5-0.8B-IQ4_XS.gguf and Qwen_Qwen3.5-0.8B-Q5_K_M.gguf), each checked by SHA-256. The GPU runs used all layers on the GPU and F16 KV cache.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

Each file got one question right that the other missed (exact McNemar p = 1.0); 20 questions are too few to separate the files. Our two projectors have the same tensors, byte for byte, as Unsloth's mmproj-F16.gguf and mmproj-BF16.gguf (unsloth/Qwen3.5-0.8B-GGUF revision 6ab4614); only general.* header fields (names, licence, tags) differ. The projectors were converted from Qwen/Qwen3.5-0.8B revision 2fc0636 with convert_hf_to_gguf.py --mmproj from llama.cpp b11889.

File-level fidelity summary

These values repeat the historical RTX 5090 measurements, not the separate CPU replay, on a text used in choosing the files; agreement measures closeness to BF16, not task accuracy.

Conversion and calibration sources

Both files were quantized with llama.cpp release b11617 (commit 14310190c, llama-quantize) from a BF16 conversion of the original made without the MTP layer, using an importance matrix. The calibration text is human-written only:

• Q4_K_M: the first 256 chunks of 512 tokens of the WikiText-2 raw training text (Salesforce/wikitext revision b08601e, wikitext-2-raw-v1, train split). This importance matrix is in the repository as Qwen3.5-0.8B-Artemis-imatrix.gguf.
• Q5_K_M: the whole WikiText-2 raw training text (4,894 chunks of 512 tokens) from the wikitext-2-raw archive. This importance matrix is not published.

WikiText-2 is licensed CC BY-SA 3.0. No WikiText text is included in the files. Both files' headers name TurinTech in general.quantized_by and the calibration set in quantize.imatrix.dataset.

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-2026-10-20

One NVIDIA RTX 5090, October 2026.

• Reference: BF16 GGUF conversion of Qwen/Qwen3.5-0.8B revision 2fc06364715b967f1860aea9cf38778875588b17, without the MTP layer, made with llama.cpp b11617 (1,516,744,672 bytes, SHA-256 e4cb6d90f531d4ed2b70785683126fb259b7276394d8442be48eaf37cd85e3dd)
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD, 99th-percentile KLD (llama.cpp's 99.0% KLD), Same top p and PPL ratio (Mean PPL(Q)/PPL(base)) are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -t 16 -ngl 99; the reference logs were written with the same command without --kl-divergence. Taken from the committed scoring script with the chunks, threads and GPU layers recorded for each run
• Hardware: one NVIDIA RTX 5090 (AMD EPYC 7V13 host)
• Settings: 16 threads, all layers on the GPU; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings on the card are for the task runs)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: the two texts were used in choosing both files, so these columns are not independent tests
• Source on this card: the table columns "KL wiki / chat", "Same next token, wiki / chat"
• Comparison files: Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q8_0.gguf, SHA-256 0ad885ffd4bb022fc4f0d33a3308fa108ef8613159d3b3a67e23abca056b7a6c; bartowski: bartowski/Qwen_Qwen3.5-0.8B-GGUF at f36b1ea49a33, Qwen_Qwen3.5-0.8B-Q5_K_M.gguf, SHA-256 7f5b4185c9a3fd54daae8925927377c80e16e3afa094ea9444b3c0a9077136ab; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q5_K_M.gguf, SHA-256 c3ef5827b322c4be08a3a26ce424460d8b37daf593356c2cc1f4e40a7ab0581b; bartowski: bartowski/Qwen_Qwen3.5-0.8B-GGUF at f36b1ea49a33, Qwen_Qwen3.5-0.8B-Q4_K_M.gguf, SHA-256 fb044e93939a70469c905781334f5de1e6c8b608ced6cbc8c9249bd4127d9526; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-UD-Q4_K_XL.gguf, SHA-256 3177ebd67afe4438374da19e690bc1b98756f7e0fea9240e1be404336156a7b5; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q4_K_M.gguf, SHA-256 bd258782e35f7f458f8aced1adc053e6e92e89bc735ba3be89d38a06121dc517; bartowski: bartowski/Qwen_Qwen3.5-0.8B-GGUF at f36b1ea49a33, Qwen_Qwen3.5-0.8B-IQ4_XS.gguf, SHA-256 0208d8c9491848f83fdd9b125fdcdb7495f4656314f323e5c78da8ae6c5d9426

Protocol rtx5090-2026-10-20, text wiki (WikiText-2 test):

Protocol rtx5090-2026-10-20, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Protocol cpu-probe-2026-10-15

Fresh probe, 15 October 2026 (workstation CPU, CPU only, 8 threads).

• Reference: Unsloth's public BF16 GGUF of Qwen/Qwen3.5-0.8B, unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e2023f6e3c1baea90a8f0fe38ab64d0, file Qwen3.5-0.8B-BF16.gguf (1,516,744,736 bytes, SHA-256 cedf89af31c9041b601fa58303285bc46d99c51baee1b13f5e919626ca526ee5)
• Text probe6: Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats; chunks: 2 per document; context (tokens): 256; text SHA-256: prose-literature 328e88173fa5ef54cdd72c74ded2968182b4d1e79e89f7dc5ef991c2311740f2; prose-space 872a3d009605ccf22deed05eb9d8eb378d6aad129cd1d26c609f983bdf2a3a53; code-panel 8a86fcd63e889dfffa87a6b5edfe7525f54ea2c2063bb653992e6299e1becd8e; code-runner 41c5af961875d9b5c3e91b82aa3e89c5808408c39293259ee4b402eda07585e1; chat-debug e490eafde70ac0aa94c71bb4b28bc3efaa344d33b4e68becafaf9f022e2eb724; chat-travel 80fbaecbad00bee59eaa569a21d12deea7d6db4635b3e1e887a2e0633ae5102e
• Tool: llama-perplexity --kl-divergence on each of six documents, two chunks of 256 tokens each, against the reference's saved log-probabilities; the value is the plain mean of the six documents' Mean KLD values
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> --kl-divergence-base <reference-document.kld> --kl-divergence -c 256 --chunks 2 -ngl 0 -t 8 for each document; the reference logs were written with llama-perplexity -m <BF16.gguf> -f <document> --save-all-logits <reference-document.kld> -c 256 --chunks 2 -ngl 0 -t 8
• Hardware: workstation CPU, CPU only, 8 threads
• Settings: llama-perplexity defaults otherwise; no flash attention flag
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: the six documents were frozen before the comparisons and were not used for calibration or for choosing files
• Source on this card: the table column "Mean KL ours / public"
• Comparison files: Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q4_K_M.gguf, SHA-256 bd258782e35f7f458f8aced1adc053e6e92e89bc735ba3be89d38a06121dc517; bartowski: bartowski/Qwen_Qwen3.5-0.8B-GGUF at f36b1ea49a33, Qwen_Qwen3.5-0.8B-Q4_K_M.gguf, SHA-256 fb044e93939a70469c905781334f5de1e6c8b608ced6cbc8c9249bd4127d9526; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-UD-Q4_K_XL.gguf, SHA-256 3177ebd67afe4438374da19e690bc1b98756f7e0fea9240e1be404336156a7b5; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q5_K_S.gguf, SHA-256 a201a5899049a7f7845f910ad803574787b4474c3daffc0d33e8688f9114880f

Protocol cpu-probe-2026-10-15, text probe6 (Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats):

Protocol cpu-replay-2026-10-16

CPU replay of the original texts, 16 October 2026.

• Reference: Unsloth's public BF16 GGUF of Qwen/Qwen3.5-0.8B, unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e2023f6e3c1baea90a8f0fe38ab64d0, file Qwen3.5-0.8B-BF16.gguf (1,516,744,736 bytes, SHA-256 cedf89af31c9041b601fa58303285bc46d99c51baee1b13f5e919626ca526ee5); the historical producer BF16 used for the RTX 5090 table is a different conversion
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD, 99th-percentile KLD (llama.cpp's 99.0% KLD), Same top p and PPL ratio (Mean PPL(Q)/PPL(base)) are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190c); llama-perplexity SHA-256 1c06240ed8594fd377d655aef2dab0865431e3e779c06638474c96b38e6d74a0
• Command: llama-perplexity -m <file.gguf> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -ngl 0 -t 8
• Hardware: workstation CPU, CPU only, 8 threads
• Settings: llama-perplexity defaults otherwise; no flash attention flag
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the bartowski and Unsloth files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: the same two texts were used in choosing the files; they were rebuilt independently to matching SHA-256 hashes for this replay
• Source on this card: the table columns "Wiki / chat P99 token KL", "Wiki / chat mean KL", "Wiki / chat top-1 agreement"
• Comparison files: bartowski: bartowski/Qwen_Qwen3.5-0.8B-GGUF at f36b1ea49a33, Qwen_Qwen3.5-0.8B-Q4_K_M.gguf, SHA-256 fb044e93939a70469c905781334f5de1e6c8b608ced6cbc8c9249bd4127d9526; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q4_K_M.gguf, SHA-256 bd258782e35f7f458f8aced1adc053e6e92e89bc735ba3be89d38a06121dc517; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-UD-Q4_K_XL.gguf, SHA-256 3177ebd67afe4438374da19e690bc1b98756f7e0fea9240e1be404336156a7b5; Unsloth: unsloth/Qwen3.5-0.8B-GGUF at 6ab461498e20, Qwen3.5-0.8B-Q5_K_S.gguf, SHA-256 a201a5899049a7f7845f910ad803574787b4474c3daffc0d33e8688f9114880f

Protocol cpu-replay-2026-10-16, text wiki (WikiText-2 test):

Protocol cpu-replay-2026-10-16, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

File names, sizes and SHA-256 checksums

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

File names end in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick a file by tag. Each file is a per-tensor mix of K-quants on a Q4_K base (the Q4_K_M file also has a few small tensors at Q8_0 and F16), with no type making up 90% of the bytes, so the labels are size classes: the smallest standard llama.cpp quant whose file, made from the same BF16 conversion, has at least as many bits per weight. A stock Q4_K_M of this model is 5.51 bits per weight, a stock Q5_K_S 5.88 and a stock Q5_K_M 6.03. The Q5_K_M file (5.93) follows that rule. The Q4_K_M file is an exception: by the rule it would be Q5_K_S, but at 5.54 bits per weight it is 0.6% above a stock Q4_K_M and below Unsloth's Q4_K_M (5.55), the size it was built to stay under, so it keeps the Q4_K_M label. The general.file_type field inside each file matches its label. Public files with the same label can have different tensor allocations. Bartowski's Q4_K_M download also includes an unused MTP block; see the active-weight comparison above.

File checksums

Artemis

Teams use Artemis to test and improve code against their own objectives. See Artemis.

About and licence

GGUF versions of Qwen3.5 0.8B for llama.cpp, with the vision encoder as a separate projector file. The multi-token prediction (MTP) layer is not included.

Apache-2.0, the licence Qwen releases Qwen3.5 0.8B under (licence); see LICENSE and NOTICE. The importance matrices used for quantization were computed from the WikiText-2 training text (Wikipedia, CC BY-SA 3.0); no WikiText text is included in the files, including the importance-matrix file.

These files keep improving.