turintech/Qwen3.5-2B-Artemis-GGUF

TurinTech models, built with Artemis

A small Qwen3.5 2B for local tool calls, maths and instructions, with all layers on a small GPU.

• 1.395 GB download for the default file, Qwen3.5-2B-Artemis-Q5_K_M.gguf.
• Peak GPU memory 2.1 GiB on an NVIDIA RTX 5090 with an 8,192-token context; the original peaked at 4.3 GiB.
• 184 of 258 held-out BFCL live tool calls right, against 177 for bartowski's Q4_K_M (1.396 GB).

Run it

Needs llama.cpp with Qwen3.5 and HTTPS support. -hf also downloads the vision projector and loads it; add --no-mmproj for text only, which skips the 671 MB download.

llama-server -hf turintech/Qwen3.5-2B-Artemis-GGUF:Q5_K_M --port 8080 -c 8192 --jinja

Request example, images, settings and Ollama

Keep the :Q5_K_M tag: untagged llama.cpp downloads and Hugging Face's generated Ollama import select Q4_K_M instead.

The tool-calling results include greedy and sampled decoding, both with thinking off. The quickstart and the Ollama packages do not set that full configuration; set chat_template_kwargs.enable_thinking explicitly.

llama-server -m Qwen3.5-2B-Artemis-Q5_K_M.gguf --port 8080 -c 8192 --jinja

--jinja enables the chat template. In another terminal, wait for curl -s http://localhost:8080/health to return {"status":"ok"}, then send a tool request with thinking off:

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Paris right now, in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}}, "required": ["city"]}}}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

In our test run (Q5_K_M file, CPU, llama.cpp b11617) the reply's tool_calls held get_weather with {"city":"Paris","unit":"celsius"}. The tool-call response is in choices[0].message.tool_calls.

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16.

hf download turintech/Qwen3.5-2B-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m Qwen3.5-2B-Artemis-Q5_K_M.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 16 right (16 with the F16 one), against 14 for Unsloth's Q5_K_M with Unsloth's projector (see Image questions).

Ollama

Read-only registry checks on 16 October 2026 confirmed the tag aliases, model-layer SHA-256 hashes and package settings below against these HF files. This was not a fresh registry pull or inference test.

ollama run turintech/qwen3.5-2b-artemis:q5_k_m

latest selects q5_k_m. The explicit tags below use the same model bytes as the corresponding Hugging Face files.

GB is download size, not runtime memory. Packages use Ollama's Qwen3.5 renderer and tool parser, with temperature 1, top_k 20, top_p 0.95 and presence_penalty 1.5. The comparison benchmarks used llama.cpp with different sampling settings; these packages do not establish the same benchmark results in Ollama.

A separate Ollama 0.40.4 local-import smoke test covered our audited files with one tool prompt, greedy decoding, seed 20261015 and a 256-token output cap. In this test, omitting think enabled reasoning; think:false disabled it. This checks import and parser behavior on one prompt, not task accuracy or a fresh registry pull.

The packages hold the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/Qwen3.5-2B-Artemis-GGUF:Q5_K_M

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this model brought its projector along and ollama show listed vision. A local package of our Q5_K_M file and mmproj-F16.gguf also listed vision, answered 16 of the 20 chart questions and returned get_weather with Paris and celsius for the tool request above. To build that package, put both files next to a Modelfile with two FROM lines:

FROM ./Qwen3.5-2B-Artemis-Q5_K_M.gguf
FROM ./mmproj-F16.gguf
RENDERER qwen3.5
PARSER qwen3.5
PARAMETER num_ctx 8192

Choose this model

• Best for tool calls and short chat answers on a small GPU.
• Consider another file when you want the smaller download: our Q4_K_M (1.280 GB) got 189 of those tool calls, against 163 for Unsloth's Q4_K_M of the same size. No paired test separated the Q5_K_M from bartowski's file or the original.

More on choosing

On the RTX 5090 suite's independent tasks, which no recorded selection run used, one paired test survived correction: on 258 BFCL live tool calls the Q4_K_M got 189 right against 163 for Unsloth's Q4_K_M, whose replies made no tool call 37 times against 12. No other test separated our files from the BF16 original or the size-matched public files. The default Q5_K_M got 190 of 250 GSM8K problems, 69 of 280 MMLU-Pro questions, 184 live tool calls and 374 of 541 IFEval prompts right, against 189, 69, 182 and 360 for BF16. The smaller Q4_K_M followed 347 IFEval prompts. In an earlier RTX 5090 HumanEval+ check, Q5_K_M passed 73 of 164 coding tasks and Q4_K_M 70, against 81 for BF16. With all layers on an RTX 5090 the default peaked at 2.1 GiB of GPU memory.

The results do not establish overall superiority or lossless compression. See the measured trade-offs and test conditions in How it was tested.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/Qwen3.5-2B-Artemis-GGUF:Q5_K_M.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/qwen3.5-2b-artemis:<tag>.
• LM Studio's Hub entry turintech/qwen3.5-2b-artemis offers every model file above, without the importance matrix or the vision projector.

Without a tag, -hf and hf.co fetch Qwen3.5-2B-Artemis-Q4_K_M.gguf, not the default file, so always add the tag.

The vision projector is optional, for image input. There is no MTP file.

How the files compare

GB means decimal download size, not RAM or VRAM. Labels are size classes; tensor types and checksums are below.

Results

Our default file against bartowski's Q4_K_M, the closest public file in size, and the original. These tasks played no part in choosing the files.

No paired test separated the Q5_K_M from bartowski's file or the original after correction. The one difference that survives is our smaller Q4_K_M against Unsloth's Q4_K_M on tool calls, 189 against 163. How it was tested.

Correct tool calls on the 258 BFCL live_simple tasks, an independent panel that no recorded selection run used, measured on an NVIDIA RTX 5090 with llama.cpp b11617 and drawn on the full 0 to 258 scale. Our Q4_K_M got 189 right against 163 for Unsloth's Q4_K_M of the same size, the one difference that survives correction; Unsloth's file made no tool call on 37 tasks against 12 for ours. The original and Unsloth's Q8_0 got 182 and our Q5_K_M 184; no other pre-registered paired test separated our files from the original or the matched public files after correction. Every task expects a call, so the chart does not show whether a file holds back when no tool fits. See How it was tested.

Memory

The upstream architecture declares a 262,144-token context limit; this audit does not validate these GGUFs at that maximum. A separate Q4_K_M retrieval probe recovered three codes from repetitive filler at 12,067 and 24,067 input tokens, using a 32,768-token server context and seed 2026101504. This is not long-document reasoning evidence. YaRN extension and quantized KV caches were not evaluated.

Peak memory was measured on an NVIDIA RTX 5090 with llama.cpp b11617, all layers on the GPU, an 8,192-token context, F16 K and V caches and one slot. Each file ran in its own llama-server process with the prompt cache off (--cache-ram 0) and answered the first 25 items of each benchmark task. GPU memory is the peak from nvidia-smi; host memory is the server's peak resident memory, which counts the parts of the memory-mapped model file that were read.

The host figures need --cache-ram 0. The commands on this card keep llama-server's default 8 GiB prompt cache, which lives in host memory: over the full benchmark, host memory grew to 9.4 to 9.6 GiB for every quantized file as the cache filled (9.4 GiB for the default Q5_K_M), while GPU memory stayed the same. Add --cache-ram 0 to the command to stay near the table's host figures. CPU-only memory was not measured. The context length and other runtime settings change these numbers.

With the projector loaded (mmproj-F16.gguf, RTX 5090, llama.cpp b11889, -ngl 99 -c 8192 -np 1 --cache-ram 0), 3.1 GiB of GPU memory was in use after the 20 chart questions.

How it was tested

Measurements, comparisons and limitations

NVIDIA RTX 5090 suite, October 2026

In October 2026 every file below ran on one NVIDIA RTX 5090 (AMD EPYC 7V13 host) with llama.cpp b11617 (14310190c), all layers on the GPU, an 8,192-token context, flash attention on, F16 K and V caches, greedy decoding, thinking off and one slot. Every file was served with Qwen's original chat template and compared with a BF16 conversion of Qwen/Qwen3.5-2B revision 15852e8, made without the MTP layer. Each file was checked by SHA-256 before the run.

• KL divergence and next-token agreement over 60 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations.
• Native tool calling on BFCL simple (Python), all 400 tasks, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K maths, the first 250 test problems, replies capped at 768 tokens.
• MMLU-Pro, 280 questions (20 from each category), zero-shot with a direct letter answer and replies capped at 8 tokens. This is not the leaderboard protocol, and a reply without a letter counts as wrong.
• IFEval, all 541 prompts, strict prompt-level accuracy with the lm-evaluation-harness v0.4.13 checks, replies capped at 2,048 tokens.

Some of these tasks were used in choosing the published files. The candidate files were tuned on KL over these same two texts, so neither the KL nor the same-next-token columns are independent tests; the earlier comparison below reports KL on C4 and OPUS-100 text that was held out. The calibration text also includes OpenAssistant OASST2 training conversations, and its overlap with the OASST1 validation text has not been checked. BFCL tasks 0 to 199 were tuning data, and five candidates ran on tasks 200 to 399 before the two published files were picked. A GSM8K sample and an MMLU-Pro sample were split into tuning and held-out halves; their problems were not recorded, so GSM8K here is treated as used and the MMLU-Pro overlap is unknown. IFEval was first run on these files after they were chosen. To test the files on tasks that no recorded selection run used, the run added an independent panel with the same prompts, caps and checkers. These tasks are independent as far as the records go; unrecorded earlier use cannot be ruled out.

• GSM8K fresh: the last 250 test problems (rows 1069 to 1318).
• MMLU-Pro fresh: 280 other questions (20 per category, seed 20261022), none of them in the sample above. The unrecorded earlier sample could overlap with it by chance, about one question in 40.
• BFCL live_simple: all 258 tasks, user-contributed requests with one function to call. No selection record for these files uses this category; 60 of the tasks ran in a check after the files were chosen.
• IFEval, as above.

Before the run, six comparisons were fixed for this card: each of our files against the original, Q4_K_M against Unsloth's Q4_K_M, and Q5_K_M against bartowski's Q4_K_M, Unsloth's UD-Q4_K_XL and Unsloth's Q5_K_S, each on the four independent tasks. Only these 24 tests, after a Bonferroni correction for 24, can support a stated difference between two files.

"No tool call at all" counts replies to the 400 BFCL tasks that made no tool call. The texts behind the KL and same-next-token columns, the BFCL tasks and GSM8K were used in choosing the files; IFEval was not.

The paired test compares two files on the same items. It counts the items only one of the two got right and gives the chance of a split at least that uneven if the files were equally good (exact McNemar test). A large p means the test found no difference, which is not proof that the files are equal.

Each cell gives the items only the first file got right, the items only the second got right, and p, uncorrected. After the Bonferroni correction for these 24 tests, one difference remains: on live tool calls our Q4_K_M got more right than Unsloth's Q4_K_M (33 / 7, corrected p=0.001). Unsloth's file made no tool call on 37 of the 258 live tasks, against 12 for ours. Three more tests have p below 0.05 before correction and none after: our Q5_K_M against Unsloth's UD-Q4_K_XL on GSM8K fresh (0.45 corrected) and live tool calls (0.16), and against Unsloth's Q5_K_S on IFEval (0.067).

What the tables show, without a significance claim:

• Our two files differ on IFEval: the Q5_K_M followed 374 prompts and the Q4_K_M 347 (54 / 27 items only one got right, p=0.004, a comparison outside the corrected family). The original followed 360.
• On the 400 BFCL simple tasks, all used in choosing the files, our Q4_K_M got 327 right and our Q5_K_M 319, against 303 for the original and 254 for Unsloth's Q4_K_M. The files with fewer right tended to make no tool call more often, as the last column shows.
• On KL, our Q5_K_M is closer to the original than bartowski's Q4_K_M (.0073 against .0209 on WikiText-2), Unsloth's UD-Q4_K_XL (.0173) and Unsloth's Q5_K_S (.0102), and level with bartowski's larger Q5_K_M on WikiText-2 (.0073; on the chat text bartowski's file is closer, .0065 against .0068). Our Q4_K_M is closer than Unsloth's Q4_K_M (.0125 against .0242). These KL texts were tuning data for our files.
• MMLU-Pro counts at this size mix answer format with knowledge: on the fresh sample, 16 to 99 of the 280 replies per file gave no letter (54 for the original, 43 for our Q5_K_M, 68 for our Q4_K_M), and on the earlier sample 22 to 92 (51, 45 and 57).
• Unsloth's Q8_0 got the most GSM8K fresh problems right, 200 against 190 for our Q5_K_M and 189 for the original. At this size the 768-token cap cut off some GSM8K replies: on the fresh problems 27 for the original, 20 for our Q5_K_M and 21 for our Q4_K_M.
• Every live_simple task expects a call, so the panel does not test whether a file holds back when no tool fits.

Original GTX 1080 Ti comparison

These are controlled llama.cpp comparisons with the same Qwen chat template and thinking off for every file, not tests of each publisher's default application settings. A reply without a parsed tool call counts as wrong; that alone does not show whether the cause was generation, truncation or parsing.

Every file was compared with a 3.776 GB BF16 conversion of Qwen/Qwen3.5-2B revision 15852e8c16360a2fea060d615a32b45270f8a8fc, without the MTP layer. The original comparisons in this section used an NVIDIA GTX 1080 Ti and llama.cpp b11540 (commit 1e6f04a, CUDA).

Data was split into tuning halves used to build and choose files, and held-out halves run at the end, once greedily and once sampled at temperature 0.7, top_p 0.95, with a fixed per-item seed. Five of our files ran on the held-out halves (four candidates and a rebuilt 1.34 GB file that is not published); the two published files were picked to match the sizes of the two public Q4_K_M files.

• KL divergence to BF16: C4 validation, 40 chunks of 1,024 tokens, and OPUS-100 test, 30 chunks of 1,024 tokens, on the held-out text.
• Native tool calling: BFCL v4 through llama-server's OpenAI-style tools API with --jinja, scored with BFCL's AST checker. Held-out tasks are the second half of each category: simple items 200 to 399, and items 100 to 199 of multiple, parallel and parallel_multiple, 500 in total. The first halves were tuning data. "Same calls as BF16" is the share of tasks with the same call as the original, not task accuracy.
• GSM8K and MMLU-Pro were also split into halves: the first halves were tuning data; the second, held-out halves contained 128 problems and 140 questions respectively.

In the table, g / s means greedy / sampled. Counts are correct answers, except the explicitly labelled same-call percentages.

KL and next-token agreement measure closeness to the BF16 original, including its mistakes, not whether an answer or tool call is correct. This card reports mean KL, but not KL tails or perplexity.

Our MMLU-Pro subset uses a local setup (letter-only answers) and is not comparable to Qwen's published MMLU-Pro results.

Paired exact McNemar tests count tasks only one file got right. Q4_K_M vs Unsloth Q4_K_M: 65/14 greedy and 68/16 sampled, both p < 1e-4. Q5_K_M vs bartowski Q4_K_M: 21/16 greedy (p = 0.51) and 37/23 sampled (p = 0.09), neither significant.

The paired tests describe these fixed files on this task set; their p-values do not account for choosing between candidate files. Not detecting a difference is not evidence that two files are equal.

Separate hardware re-check

This re-check reuses BFCL task IDs from the earlier panel, so it is not an independent task set. A separate check on another machine (NVIDIA GB10, llama.cpp b11617, Unsloth's BF16 as the reference) re-ran the published files' tensors on BFCL simple items 200 to 399 only (greedy, thinking off, four server slots) and measured KL on eight short synthetic documents never used for calibration or tuning. The synthetic documents vary one narrative template, so they are a narrow check, not broad text coverage.

Fresh CPU probe, 15 October 2026

This separate check used workstation CPU, eight inference threads, llama.cpp b11617, one 8,192-token server slot, greedy decoding (seed 20261015), thinking explicitly off and the pinned public Unsloth BF16 reference. KL is the mean of six document means: two recent Wikipedia extracts, two authored code samples and two authored chats, each evaluated as two 256-token chunks. These short, fixed probes measure next-token fidelity, including the reference's mistakes.

Tool scores use 60 BFCL live_simple cases selected with seed 2026101502, from a category that no selection record for these files uses; unrecorded earlier use cannot be ruled out. The table reports exact two-sided McNemar p-values. Download sizes use decimal GB; size differences are explicit.

BF16 scored 42/60. In table order, six-document sign-flip p-values are 0.03125, 0.03125, 0.03125, 0.1875. BFCL clustered p-values are 0.50781, 1, 1, 1. The clusters are the 41 function groups defined by BFCL task IDs. After Bonferroni correction across 16 comparisons per endpoint, the KL p-values are 0.5, 0.5, 0.5, 1. None of these tool-call differences survives that correction. These are exploratory fixed-file comparisons, not evidence of general superiority or equivalence. The fourth comparator is a post-hoc coverage check added after inspecting download sizes, before inspecting its outputs.

This probe does not confirm a large tool-calling gain. Our Q5 and bartowski Q4 tied at 45/60. The KL ordering against dynamic Unsloth Q4 differs from the earlier synthetic probe above. Against the similarly sized Unsloth Q5_K_S, the mean KL gap is only 9.5%, with losses on chat-debug and prose-literature; tool scores are 45/60 versus 44/60.

Corrected authored tool diagnostic

A second tool set contains 24 authored prompts in six families, with explicit date/time, enum and exact-copy constraints. Its schemas were corrected after inspecting ambiguous v1 outputs, so this is not a blind benchmark. The rerun used greedy decoding, thinking off, a 1,024-token cap and seed 2026101506.

In the four comparison rows above, exact six-family sign-flip p-values are 1, 1, 1, 1, and Bonferroni-adjusted values across 16 diagnostic comparisons are 1, 1, 1, 1. A high score on these few related prompts does not establish general tool accuracy or equivalence.

What is not better

• Unsloth's Q4_K_M scored higher than our Q4_K_M on the originally reported MMLU-Pro subset: 77 vs 64 of 140 greedy and 69 vs 60 sampled. Its greedy 77 also exceeds BF16's 71. The reported unadjusted paired p-values are 0.035 and 0.19; the original task-level records were unavailable for independent reconstruction. These observed counts favour the public file.
• The original GSM8K counts are descriptive. The original paired records were unavailable; this audit does not establish a lack of differences or equivalence.
• The table reports correct tool-call counts for each file, including BF16 (388/372). Closeness to BF16 rests on KL and "same calls as BF16".
• On the separate synthetic KL check, Unsloth's smaller UD-Q4_K_XL (1.340 GB) was slightly closer to BF16 than our Q5_K_M (3% lower mean KL), with correct-call counts for those 200 tasks reported separately.
• The pinned GGUF headers show a 15-tensor MTP block (Q8_0 and F32), 64.7 MB in bartowski's Q4_K_M, unused by plain decoding. Unsloth's files and ours have none.
• The original tool tests covered simple, multiple, parallel and parallel_multiple calls; those benchmarks did not test follow-up turns after a tool result, thinking mode or long contexts. The separate runtime probes have narrower scope than task-accuracy benchmarks. The original tool results cover one machine and one llama.cpp build; the separate checks below use other configurations. Apple Silicon was not tested. Tool-irrelevance cases were not evaluated, so these call rates do not establish when the model should avoid calling a tool.

We make no speed claim for these files; the memory figures hold for the settings given in Memory. In one decode test on an NVIDIA RTX 3060, the Q5_K_M file generated about 3% slower than bartowski's Q4_K_M.

Public comparisons: unsloth/Qwen3.5-2B-GGUF revision f6d5376 (Qwen3.5-2B-Q4_K_M.gguf, Qwen3.5-2B-UD-Q4_K_XL.gguf) and bartowski/Qwen_Qwen3.5-2B-GGUF revision 7d26695 (Qwen_Qwen3.5-2B-Q4_K_M.gguf, Qwen_Qwen3.5-2B-Q4_K_S.gguf). The RTX 5090 suite added Qwen3.5-2B-Q5_K_S.gguf, Qwen3.5-2B-Q5_K_M.gguf and Qwen3.5-2B-Q8_0.gguf from the same Unsloth revision and Qwen_Qwen3.5-2B-Q5_K_M.gguf from the same bartowski revision, each checked by SHA-256.

Broader checks: instruction following and coding (15 October 2026)

These separate checks used an NVIDIA RTX 5090, CUDA, llama.cpp b11617, the original Qwen template and thinking off. Server context was 32,768 total across eight slots, giving 4,096 tokens per slot. HumanEval+ used one greedy sample per problem with a 768-token generation cap and evalplus sanitisation; success requires both base and plus tests to pass. IFEval used a 1,280-token generation cap, temperature zero and eight concurrent requests. IFEval covers all 541 prompts.

Q4_K_M passed 70/164 coding tasks, versus 81/164 for both BF16 and Unsloth Q4_K_M. Q5_K_M passed 73/164, versus 81/164 for BF16 and 77/164 for bartowski Q4_K_M. These observed coding deficits remain trade-offs despite the tool/fidelity results above.

These counts were checked against saved aggregate receipts; inference and per-item scoring were not independently repeated in this audit. These counts describe this harness and generation budget, separately from the earlier benchmarks. Numerical best is not statistical superiority; a nonsignificant difference is not equivalence. These reviewed outcomes must be treated as development evidence if used to steer further compression searches.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

Our file got 2 questions right that Unsloth's missed and Unsloth's none that ours missed (exact McNemar p = 0.5); 20 questions are too few to separate the files. Our two projectors have the same tensors, byte for byte, as Unsloth's mmproj-F16.gguf and mmproj-BF16.gguf (unsloth/Qwen3.5-2B-GGUF revision f6d5376); only general.* header fields (names, licence, tags) differ. The projectors were converted from Qwen/Qwen3.5-2B revision 15852e8 with convert_hf_to_gguf.py --mmproj from llama.cpp b11889.

File-level fidelity summary

These values repeat the RTX 5090 measurements above, on a text used in choosing the files; agreement measures closeness to BF16, not task accuracy.

Conversion and calibration sources

The original weights were converted to BF16 GGUF without MTP and quantized for llama.cpp.

The importance matrix used 443 chunks of 1,024 tokens of human-written text only:

• OpenAssistant/oasst2 train, revision 179dd21f: non-synthetic, best-ranked reply chains, Apache-2.0.
• Salesforce/wikitext, wikitext-103-raw-v1 train, revision b08601e0: CC BY-SA 3.0 / GFDL.
• code-search-net/code_search_net, Python validation functions, revision bd0cf261: per-repository licences. The selected-function licence inventory is still being reviewed; no calibration source text is included in the release.

The importance matrix and quantization used llama.cpp b11540 (commit 1e6f04a).

The published importance matrix is Qwen3.5-2B-Artemis-imatrix.gguf. Each model is one standalone GGUF file. The importance-matrix file is calibration data for quantization, not needed to run the model; every model file in this repository was quantized with it. No calibration text is included in any file. Headers name TurinTech in general.quantized_by, the published matrix in quantize.imatrix.file, and the calibration data in quantize.imatrix.dataset. Release header changes leave the evaluated tensor data unchanged.

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-2026-10-23

One NVIDIA RTX 5090, October 2026.

• Reference: BF16 GGUF conversion of Qwen/Qwen3.5-2B revision 15852e8c16360a2fea060d615a32b45270f8a8fc, without the MTP layer, made with llama.cpp b11617 (3,775,709,152 bytes, SHA-256 9ded50dbc63830b303fe20684f3f97c7ba7bc2aa85eaef61a015e8b5d94444bd)
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD, 99th-percentile KLD (llama.cpp's 99.0% KLD), Same top p and PPL ratio (Mean PPL(Q)/PPL(base)) are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -t 16 -ngl 99; the reference logs were written with the same command without --kl-divergence. Taken from the committed scoring script with the chunks, threads and GPU layers recorded for each run
• Hardware: one NVIDIA RTX 5090 (AMD EPYC 7V13 host)
• Settings: 16 threads, all layers on the GPU; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings on the card are for the task runs)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: the candidate files were tuned on KL over these same two texts, so these columns are not independent tests
• Source on this card: the table columns "KL chat", "Same next token, chat", "KL wiki", "Same next token, wiki"
• Comparison files: Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q4_K_M.gguf, SHA-256 aaf42c8b7c3cab2bf3d69c355048d4a0ee9973d48f16c731c0520ee914699223; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-UD-Q4_K_XL.gguf, SHA-256 0af96165ea615bea39a04118d63f0b6d35908aea850ee4a51aa6151d851b8b35; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q5_K_S.gguf, SHA-256 5503c57f038bd201e0583d64d50d4aebba2f628004834cb4d32abb141eb1bfda; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q5_K_M.gguf, SHA-256 1885b3a9195f8cc09da9a7a7a75afdc1e8d5cbf9fc4a499c3961dddea37098ac; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q8_0.gguf, SHA-256 1b04acba824817554f4ce23639bc8495ff70453b8fcb047900c731521021f2c1; bartowski: bartowski/Qwen_Qwen3.5-2B-GGUF at 7d26695454df, Qwen_Qwen3.5-2B-Q4_K_S.gguf, SHA-256 55b574899b75180d084238ffbf5d3d165c568a3d73e1edbd3c826682a986c8d4; bartowski: bartowski/Qwen_Qwen3.5-2B-GGUF at 7d26695454df, Qwen_Qwen3.5-2B-Q4_K_M.gguf, SHA-256 57a1085840f497d764a7fc5d346922dbde961efb54cc792ea81d694fd846a1d8; bartowski: bartowski/Qwen_Qwen3.5-2B-GGUF at 7d26695454df, Qwen_Qwen3.5-2B-Q5_K_M.gguf, SHA-256 3f50d5f830873c22d33558f69eb2c34aa59858f37e7a58a982aae88bb38ebe88

Protocol rtx5090-2026-10-23, text wiki (WikiText-2 test):

Protocol rtx5090-2026-10-23, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Protocol gtx1080ti-2026-10-15

NVIDIA GTX 1080 Ti, 15 October 2026.

• Reference: BF16 GGUF conversion of Qwen/Qwen3.5-2B revision 15852e8c16360a2fea060d615a32b45270f8a8fc, without the MTP layer (3.776 GB)
• Text c4: C4 validation, held-out half; split: validation; chunks: 40; context (tokens): 1024
• Text opus: OPUS-100 test, held-out half; split: test; chunks: 30; context (tokens): 1024
• llama.cpp build: b11540 (commit 1e6f04a)
• Hardware: NVIDIA GTX 1080 Ti
• Measured by: TurinTech measured every file in this table on the same machine; our two files were measured as pre-release builds that differ from the published files only in header metadata (128 bytes), with the same tensor data
• Selection: held-out halves of the C4 and OPUS-100 data, run at the end after the files had been built and chosen on the tuning halves
• Source on this card: the table column "KL C4 / OPUS"
• Notes: a separate GB10 re-check on the card prints only relative changes in mean KL, so it has no table here
• Comparison files: bartowski: bartowski/Qwen_Qwen3.5-2B-GGUF at 7d26695454df, Qwen_Qwen3.5-2B-Q4_K_M.gguf, SHA-256 57a1085840f497d764a7fc5d346922dbde961efb54cc792ea81d694fd846a1d8; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-UD-Q4_K_XL.gguf, SHA-256 0af96165ea615bea39a04118d63f0b6d35908aea850ee4a51aa6151d851b8b35; bartowski: bartowski/Qwen_Qwen3.5-2B-GGUF at 7d26695454df, Qwen_Qwen3.5-2B-Q4_K_S.gguf, SHA-256 55b574899b75180d084238ffbf5d3d165c568a3d73e1edbd3c826682a986c8d4; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q4_K_M.gguf, SHA-256 aaf42c8b7c3cab2bf3d69c355048d4a0ee9973d48f16c731c0520ee914699223

Protocol gtx1080ti-2026-10-15, text c4 (C4 validation, held-out half):

Protocol gtx1080ti-2026-10-15, text opus (OPUS-100 test, held-out half):

Protocol cpu-probe-2026-10-15

Fresh probe, 15 October 2026 (workstation CPU, CPU only, 8 threads).

• Reference: Unsloth's public BF16 GGUF of Qwen/Qwen3.5-2B, unsloth/Qwen3.5-2B-GGUF at f6d5376be1edb4d416d56da11e5397a961aca8ae, file Qwen3.5-2B-BF16.gguf (3,775,709,216 bytes, SHA-256 dc11be2ca4519954a3e4b013ef59497b5522209987d67891851113eda89619f1)
• Text probe6: Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats; chunks: 2 per document; context (tokens): 256; text SHA-256: prose-literature 328e88173fa5ef54cdd72c74ded2968182b4d1e79e89f7dc5ef991c2311740f2; prose-space 872a3d009605ccf22deed05eb9d8eb378d6aad129cd1d26c609f983bdf2a3a53; code-panel 8a86fcd63e889dfffa87a6b5edfe7525f54ea2c2063bb653992e6299e1becd8e; code-runner 41c5af961875d9b5c3e91b82aa3e89c5808408c39293259ee4b402eda07585e1; chat-debug e490eafde70ac0aa94c71bb4b28bc3efaa344d33b4e68becafaf9f022e2eb724; chat-travel 80fbaecbad00bee59eaa569a21d12deea7d6db4635b3e1e887a2e0633ae5102e
• Tool: llama-perplexity --kl-divergence on each of six documents, two chunks of 256 tokens each, against the reference's saved log-probabilities; the value is the plain mean of the six documents' Mean KLD values
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> --kl-divergence-base <reference-document.kld> --kl-divergence -c 256 --chunks 2 -ngl 0 -t 8 for each document; the reference logs were written with llama-perplexity -m <BF16.gguf> -f <document> --save-all-logits <reference-document.kld> -c 256 --chunks 2 -ngl 0 -t 8
• Hardware: workstation CPU, CPU only, 8 threads
• Settings: llama-perplexity defaults otherwise; no flash attention flag
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Unsloth and bartowski files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Selection: the six documents were frozen before the comparisons and were not used for calibration or for choosing files
• Source on this card: the table column "Mean KL ours / public"
• Comparison files: Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q4_K_M.gguf, SHA-256 aaf42c8b7c3cab2bf3d69c355048d4a0ee9973d48f16c731c0520ee914699223; bartowski: bartowski/Qwen_Qwen3.5-2B-GGUF at 7d26695454df, Qwen_Qwen3.5-2B-Q4_K_M.gguf, SHA-256 57a1085840f497d764a7fc5d346922dbde961efb54cc792ea81d694fd846a1d8; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-UD-Q4_K_XL.gguf, SHA-256 0af96165ea615bea39a04118d63f0b6d35908aea850ee4a51aa6151d851b8b35; Unsloth: unsloth/Qwen3.5-2B-GGUF at f6d5376be1ed, Qwen3.5-2B-Q5_K_S.gguf, SHA-256 5503c57f038bd201e0583d64d50d4aebba2f628004834cb4d32abb141eb1bfda

Protocol cpu-probe-2026-10-15, text probe6 (Six fixed documents: two recent Wikipedia extracts, two authored code files and two authored chats):

File names, sizes and SHA-256 checksums

File names

Bits per weight uses tensor payload bytes, excluding GGUF metadata.

Labels are size classes, not a description of every tensor: the smallest standard label whose stock file from the same BF16 has at least as many bits per weight, except where stated below.

Stock llama-quantize dry runs from the same BF16, llama.cpp b11540 (commit 1e6f04a), give IQ4_XS 5.06, Q4_K_S 5.11, Q4_K_M 5.37, Q5_K_S 5.79 and Q5_K_M 5.95 bits per weight. Our Q5_K_M (5.88) follows the size-class rule. Our Q4_K_M (5.39) is an exception: 0.4% above stock Q4_K_M, but smaller than Unsloth's Q4_K_M (1,280,835,840 bytes), the file it was built to match. It keeps the Q4_K_M label. Header general.file_type is 17 (Q5_K_M) or 15 (Q4_K_M).

File checksums

About and licence

GGUF versions of Qwen3.5 2B for llama.cpp, with the vision encoder as a separate projector file. The multi-token prediction (MTP) layer is not included.

Apache-2.0, as released by Qwen (licence). See LICENSE and NOTICE. This is an independent TurinTech conversion, not affiliated with or endorsed by Alibaba Cloud or the Qwen team.

Anyone serving these files to others is asked to mention TurinTech (turintech.ai) as the source of the quantization. This is a request, not a licence condition.

These files keep improving.