turintech/Qwen3.5-4B-Artemis-GGUF

TurinTech models, built with Artemis

Qwen3.5 4B for local chat and tool calls, with all layers on a small GPU.

• 2.714 GB download for the default file, Qwen3.5-4B-Artemis-Q4_K_M.gguf.
• Peak GPU memory 3.5 GiB on an NVIDIA RTX 5090 with an 8,192-token context; the original peaked at 8.9 GiB.
• Its next token matches the original's on 93.5% of WikiText-2 tokens, against 92.7% for Unsloth's Q4_K_M (2.741 GB). WikiText-2 also chose the file.

Run it

Needs llama.cpp with Qwen3.5 and HTTPS support. -hf also downloads the vision projector and loads it; add --no-mmproj for text only, which skips the 676 MB download.

llama-server -hf turintech/Qwen3.5-4B-Artemis-GGUF:Q4_K_M --port 8080 -c 8192 --jinja

Request example, images, settings and Ollama

The tool-calling results use greedy decoding with thinking off. The quickstart and the Ollama packages do not use that full configuration; set chat_template_kwargs.enable_thinking explicitly.

llama-server -m Qwen3.5-4B-Artemis-Q4_K_M.gguf --port 8080 -c 8192 --jinja

--jinja enables the chat template. In another terminal, wait for curl -s http://localhost:8080/health to return {"status":"ok"}, then send a tool request with thinking off:

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

The tool-call response is in `choices[0].message.tool_calls`.

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16.

hf download turintech/Qwen3.5-4B-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m Qwen3.5-4B-Artemis-Q4_K_M.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 14 right (15 with the F16 one), against 15 for Unsloth's Q4_K_M with Unsloth's projector (see Image questions).

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-4b-artemis:q4_k_m

latest selects q4_k_m. The explicit tags below use the same model bytes as the corresponding Hugging Face files.

GB is download size, not runtime memory. Packages use Ollama's Qwen3.5 renderer and tool parser, with temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5. The comparison benchmarks used llama.cpp with different sampling settings; these packages do not establish the same benchmark results in Ollama.

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull.

The packages hold the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/Qwen3.5-4B-Artemis-GGUF:Q4_K_M

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this model brought its projector along and ollama show listed vision. A local package of our Q4_K_M file and mmproj-F16.gguf also listed vision, answered 15 of the 20 chart questions and returned get_weather with Paris and celsius for the tool request above. To build that package, put both files next to a Modelfile with two FROM lines:

FROM ./Qwen3.5-4B-Artemis-Q4_K_M.gguf
FROM ./mmproj-F16.gguf
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER num_ctx 8192

Choose this model

• Best for chat and tool calls on a small GPU; on 258 BFCL live tool calls it got 204, the same as the original.
• Consider another file when instruction following matters most: its IFEval count, 431 of 541, is the lowest of the eleven files measured, and bartowski's Q4_K_M (3.013 GB) followed 444.

More on choosing

On the RTX 5090 suite's independent tasks, which no recorded selection run used, no paired test separated any of our three files from the BF16 original or from the size-matched public files after correction. The default Q4_K_M got 231 of 250 GSM8K problems, 125 of 280 MMLU-Pro questions, 204 of 258 BFCL live tool calls and 431 of 541 IFEval prompts right, against 238, 127, 204 and 446 for BF16 and 233, 129, 202 and 434 for Unsloth's Q4_K_M. On 9 of the 10 GSM8K problems BF16 got right and the default missed, the default's reply hit the 768-token cap. Its IFEval count is the lowest of the eleven files measured. On the 400 BFCL simple tasks, all used in choosing the files, ours got 358 to 360 right against 362 for BF16 and 369 to 372 for four public files. With all layers on an RTX 5090 the default peaked at 3.5 GiB of GPU memory.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/Qwen3.5-4B-Artemis-GGUF:Q4_K_M.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/qwen3.5-4b-artemis:<tag>.
• LM Studio's Hub entry turintech/qwen3.5-4b-artemis offers every model file above, without the importance matrix or the vision projector.

The vision projector is optional, for image input. There is no MTP file.

How the files compare

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Results

Our default file against Unsloth's Q4_K_M, the closest public file in size, and the original. Same next token is on WikiText-2, the text the files were chosen on; the other columns are tasks no selection run used.

The files were chosen on closeness over WikiText-2, so that column is not an independent test. On the independent tasks the default sits at or below the original, and its IFEval count is the lowest of the eleven files measured. How it was tested.

KL divergence from the BF16 original over 60 chunks of the WikiText-2 test set, measured on an NVIDIA RTX 5090 with llama.cpp b11617; lower is closer, and the original is the reference at zero. The files were chosen on KL over this text, so it is not an independent test, and KL is not task accuracy: on the independent tasks no paired test separated our files from the original or the matched public files, and the default's counts there were at or below the original's, with the lowest IFEval count of the eleven files. Our Q4_K_M has about three quarters of the KL of Unsloth's Q4_K_M on this run (about nine tenths in the earlier Apple M3 Max run), and our Q5_K_S less than Unsloth's UD-Q4_K_XL of the same size; Unsloth's Q5_K_S is about level with our Q5_K_M, and the larger Q5_K_M files from Unsloth and bartowski are closer than any of ours. See How it was tested.

Memory

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated.

Peak memory was measured on an NVIDIA RTX 5090 with llama.cpp b11617, all layers on the GPU, an 8,192-token context, F16 K and V caches and one slot. Each file ran in its own llama-server process with the prompt cache off (--cache-ram 0) and answered the first 25 items of each benchmark task. GPU memory is the peak from nvidia-smi; host memory is the server's peak resident memory, which counts the parts of the memory-mapped model file that were read.

The host figures need --cache-ram 0. The commands on this card keep llama-server's default 8 GiB prompt cache, which lives in host memory: over the full benchmark, host memory grew to 9.3 to 9.6 GiB for every quantized file as the cache filled (9.3 GiB for the default Q4_K_M), while GPU memory stayed the same. Add --cache-ram 0 to the command to stay near the table's host figures. CPU-only memory was not measured. The context length and other runtime settings change these numbers.

With the projector loaded (mmproj-F16.gguf, RTX 5090, llama.cpp b11889, -ngl 99 -c 8192 -np 1 --cache-ram 0), 4.4 GiB of GPU memory was in use after the 20 chart questions.

How it was tested

Measurements, comparisons and limitations

NVIDIA RTX 5090 suite, October 2026

In October 2026 every file below ran on one NVIDIA RTX 5090 (AMD EPYC 7V13 host) with llama.cpp b11617 (14310190c), all layers on the GPU, an 8,192-token context, flash attention on, F16 K and V caches, greedy decoding, thinking off and one slot. Every file was served with Qwen's original chat template and compared with an 8.42 GB BF16 conversion of Qwen/Qwen3.5-4B revision 851bf6e, made without the MTP layer. Each file was checked by SHA-256 before the run.

• KL divergence and next-token agreement over 60 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations.
• Native tool calling on BFCL simple (Python), all 400 tasks, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K maths, the first 250 test problems, replies capped at 768 tokens.
• MMLU-Pro, 280 questions (20 from each category), zero-shot with a direct letter answer and replies capped at 8 tokens. This is not the leaderboard protocol, and a reply without a letter counts as wrong.
• IFEval, all 541 prompts, strict prompt-level accuracy with the lm-evaluation-harness v0.4.13 checks, replies capped at 2,048 tokens.

Some of these tasks were used in choosing the published files. Candidate files were ranked on KL over the same two texts, and the OASST1 validation KL also informed adding OASST1 training text to the calibration, so neither the KL nor the same-next-token columns are independent tests. All 400 BFCL tasks were run before the choice; one candidate was left out partly on its result on tasks 200 to 399. GSM8K problems 0 to 249 were measured in both selection runs. An MMLU-Pro sample of the same size was also run at selection time; its questions were not recorded, so its overlap with this sample is unknown. IFEval was first run on these files after they were chosen. To test the files on tasks that no recorded selection run used, the run added an independent panel with the same prompts, caps and checkers. These tasks are independent as far as the records go; unrecorded earlier use cannot be ruled out.

• GSM8K fresh: the last 250 test problems (rows 1069 to 1318).
• MMLU-Pro fresh: 280 other questions (20 per category, seed 20261022), none of them in the sample above. The unrecorded earlier sample could overlap with it by chance, about one question in 40.
• BFCL live_simple: all 258 tasks, user-contributed requests with one function to call. No selection record for these files uses this category; 60 of the tasks ran in a check after the files were chosen.
• IFEval, as above.

Before the run, seven comparisons were fixed for this card: each of our files against the original, Q4_K_M against Unsloth's Q4_K_M, Q5_K_S against Unsloth's UD-Q4_K_XL, and Q5_K_M against bartowski's Q4_K_M and Unsloth's Q5_K_S, each on the four independent tasks. Only these 28 tests, after a Bonferroni correction for 28, can support a stated difference between two files.

"No tool call at all" counts replies to the 400 BFCL tasks that made no tool call. The texts behind the KL and same-next-token columns, the BFCL tasks and GSM8K were used in choosing the files; IFEval was not.

The paired test compares two files on the same items. It counts the items only one of the two got right and gives the chance of a split at least that uneven if the files were equally good (exact McNemar test). A large p means the test found no difference, which is not proof that the files are equal.

Each cell gives the items only the first file got right, the items only the second got right, and p, uncorrected. After the Bonferroni correction for these 28 tests, none remains, so the card claims no task difference between our files and the original or the matched public files. The two smallest p values, Q4_K_M against the original on IFEval and Q5_K_M against Unsloth's Q5_K_S on live tool calls, are 1.00 after correction.

What the tables show, without a significance claim:

• Our Q4_K_M followed the fewest IFEval prompts of the eleven files, 431 against 446 for the original and 434 for Unsloth's Q4_K_M. Our larger files followed 437 and 442.
• On the 400 BFCL simple tasks, all used in choosing the files, ours got 358 to 360 right against 362 for the original; Unsloth's UD-Q4_K_XL, Q5_K_S and Q5_K_M and bartowski's Q4_K_M got 369 to 372.
• On KL, our Q4_K_M is closer to the original than Unsloth's Q4_K_M (.0191 against .0248 on WikiText-2) and further than Unsloth's larger UD-Q4_K_XL (.0169). Our Q5_K_M is closer than bartowski's Q4_K_M (.0094 against .0194) and about level with Unsloth's Q5_K_S (.0096). Unsloth's Q5_K_M (3.144 GB) and bartowski's Q5_K_M (3.444 GB) are closer than any of ours.
• bartowski's Q4_K_M got the most MMLU-Pro fresh questions right, 135 against 125 to 130 for ours and 127 for the original. Per file, at most 4 of the 280 replies on the earlier sample and 5 on the fresh sample gave no letter.
• The 768-token cap cut off more GSM8K fresh replies from our Q4_K_M and Q5_K_M (15 and 13) than from the original (4). Of the 10 problems only the original got right against our Q4_K_M, 9 were cut off in ours.
• Every file called a tool on almost every BFCL task: at most 11 of 400 replies, and 2 to 9 of the 258 live tasks, made no call. Every live_simple task expects a call, so the panel does not test whether a file holds back when no tool fits.

Original Apple M3 Max comparison

These are controlled llama.cpp comparisons with the same Qwen chat template and thinking off for every file, not tests of each publisher's default application settings. A reply without a parsed tool call counts as wrong; that alone does not show whether the cause was generation, truncation or parsing.

Every file was compared with a BF16 conversion of Qwen/Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. The producer report lists an 8.67 GB BF16 reference and separately describes a quantization base converted with --no-mtp, reported to have identical logits to the full BF16. The report does not establish that the 8.67 GB figure describes the MTP-free artifact. The original measurements in this subsection used one Apple M3 Max (Metal) and llama.cpp b11617 (commit 14310190c). The historical BF16 artifact identity and logit-equivalence assertion were not independently established by this audit.

• KL and same top token as BF16: WikiText-2 test and OASST1 validation (human chat), 60 chunks of 512 tokens each.
• BFCL v3 simple native tool calling: llama-server's tools API, greedy, thinking off, BFCL's ast_checker, and Qwen's original chat template for every file. Items 0 to 199 were development data; items 200 to 399 were reserved initially and run once at the end.
• GSM8K: 250 items. MMLU-Pro: 280 questions, 20 per category.
• Run 2 rebuilt the same files with a second, non-overlapping calibration slice. A single value in the table is not a repeated measurement.

KL and next-token agreement measure closeness to the BF16 original, including its mistakes, not whether an answer or tool call is correct. This card reports mean KL, but not KL tails or perplexity.

Our MMLU-Pro subset uses a local setup (letter-only answers) and is not comparable to Qwen's published MMLU-Pro results.

Four of our files ran on the selection-exposed tasks; one was not published partly because its tool calls trailed BF16 (only ours right / only BF16 right: 2/10, p = 0.04). The selection-exposed results therefore influenced which files were published.

Repeat measurements with a second calibration slice

Run 2 rebuilt our files with a different calibration slice. These repeats check reproducibility; public comparator files were not rerun. They are not a separate leaderboard.

The table reports run 1 KL for each file and corpus. Run 2 KL for Artemis Q4_K_M, Q5_K_S and Q5_K_M is .0179/.0324, .0107/.0221 and .0078/.0169 respectively. Bartowski Q4_K_M records .0177/.0318 in run 1. These are file-level fidelity measurements, not task accuracy.

Paired exact McNemar tests on selection-exposed BFCL (only ours right / only theirs right): Q4_K_M vs Unsloth Q4_K_M, 5/5 (p = 1.0); Q5_K_S vs UD-Q4_K_XL, 4/3 (p = 1.0); Q5_K_M vs bartowski Q4_K_M, 2/4 (p = 0.69). Against BF16: Q4_K_M 5/7 (p = 0.77), Q5_K_S 4/1 (p = 0.38), Q5_K_M 5/2 (p = 0.45).

The paired tests describe these fixed files on this task set; their p-values do not account for choosing between candidate files. Not detecting a difference is not evidence that two files are equal.

Same per-item BFCL correctness outcome as BF16 over development plus selection-exposed (400 tasks): Q5_K_S 395, Q5_K_M 393, Q4_K_M 381, Unsloth UD-Q4_K_XL 387, Unsloth Q4_K_M 382, bartowski Q4_K_M 382. This is correctness agreement, not exact-call agreement: raw calls were not stored, so exact-call agreement was not measured.

Fresh CPU probe, 15 October 2026

This separate check used workstation CPU, eight inference threads, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, from a category that no selection record for these files uses; unrecorded earlier use cannot be ruled out. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

BF16 scored 49/60. In table order, six-document sign-flip p-values are 0.3125, 0.03125, 0.03125, 0.9375. BFCL clustered p-values are 1, 1, 0.25, 0.5. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 1, 0.5, 0.5, 1. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

Our Q4 mean KL is only modestly lower than plain Unsloth Q4, with losses on the code-runner and chat-debug documents. This is an average over these six fixed samples, not a win on every text. Our Q5_K_M scored 46/60 against bartowski Q4 at 49/60 (0/3 discordances, exact p=0.25), despite lower mean KL. Our Q4/Q5_K_S/Q5_K_M scores were 48/49/46, so the larger file did not improve this tool sample. Against Unsloth Q5_K_S, our Q5_K_M mean KL is only 1.9% lower and is higher on four of six documents (chat-travel, code-panel, prose-literature and prose-space); tool scores are 46/60 versus 48/60, with exact p=0.5.

Corrected authored tool diagnostic

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

In the four comparison rows above, exact six-family sign-flip p-values are 1, 1, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

What is not better

• No significant tool-calling differences were detected in the original Metal panel. Bartowski Q4_K_M got 185 selection-exposed tasks right against our Q5_K_M's 183. Unsloth UD-Q4_K_XL got 182 against our Q4_K_M's 178, at a larger size.
• No task gain or equivalence is claimed. The evaluated staging files' paired BFCL comparisons against BF16 were not significant; GSM8K and MMLU-Pro cover small subsets.
• The pinned GGUF headers show a 15-tensor MTP block (Q8_0 and F32), 128.2 MB in bartowski's Q4_K_M, unused by plain decoding. Unsloth's files and ours have none. Bartowski's Q4_K_M is 2.885 GB without that block.

The original tool-calling comparison covered only single-function BFCL simple calls with thinking off: not choosing among several tools, parallel or nested calls, follow-up turns after a tool result, thinking mode or long contexts.

We make no speed claim for these files; the memory figures hold for the settings given in Memory. In one decode test on an NVIDIA RTX 3060, the Q5_K_M file generated about 3% slower than bartowski's Q4_K_M.

Public files: unsloth/Qwen3.5-4B-GGUF revision e87f176479d0855a907a41277aca2f8ee7a09523 (Qwen3.5-4B-Q4_K_M.gguf, Qwen3.5-4B-UD-Q4_K_XL.gguf) and bartowski/Qwen_Qwen3.5-4B-GGUF revision 4168f45a16a1290d65a4ec0fa312ae917a4c15d6 (Qwen_Qwen3.5-4B-Q4_K_M.gguf). The RTX 5090 suite added Qwen3.5-4B-Q5_K_S.gguf, Qwen3.5-4B-Q5_K_M.gguf and Qwen3.5-4B-Q8_0.gguf from the same Unsloth revision and Qwen_Qwen3.5-4B-Q5_K_M.gguf from the same bartowski revision, each checked by SHA-256.

Separate GB10 tool check

On NVIDIA GB10 with llama.cpp b11617 and four server slots, a separate check reused BFCL simple Python items 200–399 (200 tasks). This is a hardware re-check, not an independent task set. Results must remain separate from the original Metal comparison.

Q5_K_M trailed Unsloth UD-Q4_K_XL by 8 tasks; paired only-ours/only-theirs was 1/9 (unadjusted exact McNemar p=0.02148). Against bartowski it was 3/4 (unadjusted p=1). Q4_K_M versus Unsloth Q4_K_M was 0/5 (unadjusted p=0.0625); Q5_K_S versus Unsloth UD-Q4_K_XL was 1/6 (unadjusted p=0.125). These exploratory comparisons do not establish equivalence. Tool-irrelevance cases were not evaluated, so higher call rates do not establish when a tool should be avoided.

Broader checks: instruction following and coding (15 October 2026)

These separate checks used an NVIDIA RTX 5090, CUDA, llama.cpp b11617, the original Qwen template and thinking off. Server context was 32,768 total across eight slots, giving 4,096 tokens per slot. HumanEval+ used one greedy sample per problem with a 768-token generation cap and evalplus sanitisation; success requires both base and plus tests to pass. IFEval used a 1,280-token generation cap, temperature zero and eight concurrent requests. IFEval covers the first 270 of 541 prompts, not a random full-benchmark sample.

Q4_K_M passed 131/164 coding tasks versus 128/164 for BF16 and 121/164 for Unsloth Q4_K_M, but passed 215/270 strict instruction prompts versus 222/270 and 218/270 respectively. The larger files also have mixed results; no overall quality-preservation claim follows.

These counts were checked against saved aggregate receipts; inference and per-item scoring were not independently repeated in this audit. These counts describe this harness and generation budget, separately from the earlier benchmarks. Numerical best is not statistical superiority; a nonsignificant difference is not equivalence. These reviewed outcomes must be treated as development evidence if used to steer further compression searches.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

The two files got the same questions right. Our two projectors have the same tensors, byte for byte, as Unsloth's mmproj-F16.gguf and mmproj-BF16.gguf (unsloth/Qwen3.5-4B-GGUF revision e87f176); only general.* header fields (names, licence, tags) differ. The projectors were converted from Qwen/Qwen3.5-4B revision 851bf6e with convert_hf_to_gguf.py --mmproj from llama.cpp b11889.

File-level fidelity summary

These values repeat the RTX 5090 measurements above, on a text used in choosing the files; agreement measures closeness to BF16, not task accuracy.

Conversion and calibration sources

The original weights were converted to BF16 GGUF without MTP and quantized for llama.cpp. The released files have matching tensor tables and payloads to the evaluated files, verified through retained identity receipts and current file hashes. The vision encoder is not included.

Calibration used 947 chunks of 512 tokens with llama.cpp b11617: Salesforce/wikitext wikitext-2-raw-v1 raw train (CC BY-SA 3.0 / GFDL), plus OpenAssistant/oasst1 train, English prompter-to-assistant pairs written by humans (Apache-2.0). Calibration used the OASST1 training split; chat KL used its validation split. Different splits do not establish that their text is disjoint. Validation KL informed the choice to add OASST1 training text to the calibration, so the chat KL is not an independent test of that choice. WikiText-2 calibration and evaluation use different splits of the same corpus.

The importance matrix is calibration data for quantization, not needed to run the model; the evaluated staging models used this matrix. Each model is one standalone GGUF file. No calibration text is included. The header dataset is wikitext-2-raw-v1 train + oasst1 train (human-written).

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-2026-10-23

One NVIDIA RTX 5090, October 2026.

• Reference: BF16 GGUF conversion of Qwen/Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a, without the MTP layer, made with llama.cpp b11617 (8,424,393,600 bytes, SHA-256 5c7e8135f4f06d27b3337bf585b14f3a1054072f11329840286e71459d025aa7)
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD, 99th-percentile KLD (llama.cpp's 99.0% KLD), Same top p and PPL ratio (Mean PPL(Q)/PPL(base)) are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -t 16 -ngl 99; the reference logs were written with the same command without --kl-divergence. Taken from the committed scoring script with the chunks, threads and GPU layers recorded for each run
• Hardware: one NVIDIA RTX 5090 (AMD EPYC 7V13 host)
• Settings: 16 threads, all layers on the GPU; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings on the card are for the task runs)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: candidate files were ranked on KL over the same two texts, so these columns are not independent tests
• Source on this card: the table columns "KL chat", "Same next token, chat", "KL wiki", "Same next token, wiki"
• Comparison files: Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q4_K_M.gguf, SHA-256 00fe7986ff5f6b463e62455821146049db6f9313603938a70800d1fb69ef11a4; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-UD-Q4_K_XL.gguf, SHA-256 b252c5610a42ca82d20fe2a12813e9d069eed89292907e26c783eeb0bc961bc7; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q5_K_S.gguf, SHA-256 161be121617c45ea247eae0fd4477d26f287a368b52702e3fc99c9bc52f23061; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q5_K_M.gguf, SHA-256 8814232b85594dcd46c50e5b8b29324a7efe9e746edbe8a3d1df3d3fce7aad39; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q8_0.gguf, SHA-256 10cc391b403021dd11c614679d2fd92f611c3681d29e29651b717316965d61e1; bartowski: bartowski/Qwen_Qwen3.5-4B-GGUF at 4168f45a16a1, Qwen_Qwen3.5-4B-Q4_K_M.gguf, SHA-256 13c16f426047e2de38cd075bdade4a7bcbc8c774384876f677740cda65f8a983; bartowski: bartowski/Qwen_Qwen3.5-4B-GGUF at 4168f45a16a1, Qwen_Qwen3.5-4B-Q5_K_M.gguf, SHA-256 21c4643259ce62ffd979848e5819ce03823fd9e9902279ab54cc0d572d83d33e

Protocol rtx5090-2026-10-23, text wiki (WikiText-2 test):

Protocol rtx5090-2026-10-23, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Protocol m3max-2026-10-15

Apple M3 Max, 15 October 2026.

• Reference: BF16 GGUF conversion of Qwen/Qwen3.5-4B revision 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a; whether the KL reference was the 8.67 GB conversion or the conversion without the MTP layer is not established
• Text wiki: WikiText-2 test; split: test; chunks: 60; context (tokens): 512
• Text chat: OpenAssistant validation conversations; split: validation; chunks: 60; context (tokens): 512
• llama.cpp build: b11617 (commit 14310190c), Metal
• Hardware: Apple M3 Max (Metal)
• Measured by: TurinTech measured every file in this table on the same machine; our three files were measured as pre-release builds whose tensor data matches the published files (only the header differs)
• Selection: this was the selection sweep: the candidate files were ranked on KL over these texts
• Source on this card: the table columns "KL wiki / chat", "Same top token wiki / chat %"
• Notes: the card's second-run rows for rebuilt files are not repeated here because they are not the published files
• Comparison files: Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q4_K_M.gguf, SHA-256 00fe7986ff5f6b463e62455821146049db6f9313603938a70800d1fb69ef11a4; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-UD-Q4_K_XL.gguf, SHA-256 b252c5610a42ca82d20fe2a12813e9d069eed89292907e26c783eeb0bc961bc7; bartowski: bartowski/Qwen_Qwen3.5-4B-GGUF at 4168f45a16a1, Qwen_Qwen3.5-4B-Q4_K_M.gguf, SHA-256 13c16f426047e2de38cd075bdade4a7bcbc8c774384876f677740cda65f8a983

Protocol m3max-2026-10-15, text wiki (WikiText-2 test):

Protocol m3max-2026-10-15, text chat (OpenAssistant validation conversations):

Protocol cpu-probe-2026-10-15

Fresh probe, 15 October 2026 (workstation CPU, CPU only, 8 threads).

• Reference: Unsloth's public BF16 GGUF of Qwen/Qwen3.5-4B, unsloth/Qwen3.5-4B-GGUF at e87f176479d0855a907a41277aca2f8ee7a09523, file Qwen3.5-4B-BF16.gguf (8,424,393,632 bytes, SHA-256 9e6e2841a75f503ccb330831832fd7861266e187e0dbf149a954219ccb8c197a)
• Text probe6: Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats; chunks: 2 per document; context (tokens): 256; text SHA-256: prose-literature 328e88173fa5ef54cdd72c74ded2968182b4d1e79e89f7dc5ef991c2311740f2; prose-space 872a3d009605ccf22deed05eb9d8eb378d6aad129cd1d26c609f983bdf2a3a53; code-panel 8a86fcd63e889dfffa87a6b5edfe7525f54ea2c2063bb653992e6299e1becd8e; code-runner 41c5af961875d9b5c3e91b82aa3e89c5808408c39293259ee4b402eda07585e1; chat-debug e490eafde70ac0aa94c71bb4b28bc3efaa344d33b4e68becafaf9f022e2eb724; chat-travel 80fbaecbad00bee59eaa569a21d12deea7d6db4635b3e1e887a2e0633ae5102e
• Tool: llama-perplexity --kl-divergence on each of six documents, two chunks of 256 tokens each, against the reference's saved log-probabilities; the value is the plain mean of the six documents' Mean KLD values
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> --kl-divergence-base <reference-document.kld> --kl-divergence -c 256 --chunks 2 -ngl 0 -t 8 for each document; the reference logs were written with llama-perplexity -m <BF16.gguf> -f <document> --save-all-logits <reference-document.kld> -c 256 --chunks 2 -ngl 0 -t 8
• Hardware: workstation CPU, CPU only, 8 threads
• Settings: llama-perplexity defaults otherwise; no flash attention flag
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Source on this card: the table column "Mean KL ours / public"
• Comparison files: Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q4_K_M.gguf, SHA-256 00fe7986ff5f6b463e62455821146049db6f9313603938a70800d1fb69ef11a4; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-UD-Q4_K_XL.gguf, SHA-256 b252c5610a42ca82d20fe2a12813e9d069eed89292907e26c783eeb0bc961bc7; bartowski: bartowski/Qwen_Qwen3.5-4B-GGUF at 4168f45a16a1, Qwen_Qwen3.5-4B-Q4_K_M.gguf, SHA-256 13c16f426047e2de38cd075bdade4a7bcbc8c774384876f677740cda65f8a983; Unsloth: unsloth/Qwen3.5-4B-GGUF at e87f176479d0, Qwen3.5-4B-Q5_K_S.gguf, SHA-256 161be121617c45ea247eae0fd4477d26f287a368b52702e3fc99c9bc52f23061

Protocol cpu-probe-2026-10-15, text probe6 (Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats):

File names, sizes and SHA-256 checksums

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

Labels are size classes: the smallest standard label whose stock file from the same BF16 has at least as many bits per weight, except where stated below. Stock llama-quantize dry runs from a BF16 of the same model, llama.cpp b11540, give IQ4_XS 4.79, Q4_K_S 4.86, Q4_K_M 5.13, Q5_K_S 5.67 and Q5_K_M 5.83 bits per weight.

Our Q5_K_S (5.50) and Q5_K_M (5.71) follow that rule. Q4_K_M (5.14) is an exception: 0.19% above stock Q4_K_M, but smaller than Unsloth's Q4_K_M (2,740,937,888 bytes) and bartowski's Q4_K_M (3.013 GB), so it keeps Q4_K_M. Header general.file_type is 15 (Q4_K_M), 16 (Q5_K_S) or 17 (Q5_K_M).

File checksums

Artemis

Teams use Artemis to test and improve code against their own objectives. See Artemis.

About and licence

GGUF versions of Qwen3.5 4B for llama.cpp, with the vision encoder as a separate projector file. The multi-token prediction (MTP) layer is not included.

Apache-2.0, as released by Qwen (licence). See LICENSE and NOTICE. This is an independent TurinTech conversion, not affiliated with or endorsed by Alibaba Cloud or the Qwen team.

These files keep improving.