turintech/d3-flash-Artemis-GGUF

TurinTech models, built with Artemis

d3-flash for typed decisions about text on an 8 GB GPU, or about images on a larger one. It answers typed questions with a probability for each answer, in one pass.

• 4.87 GB download for the default file, d3-flash-Artemis-IQ4_XS.gguf.
• Peak GPU memory 6.1 GiB for text and 8.3 GiB with images on an NVIDIA RTX 5090; the original needs 17.0 GiB.
• Same answer as the original on 94.5% of 2,000 test decisions, against 95.0% for ggml-org's Q4_K_M (6.60 GB); 0 confident answers changed. These decisions also chose the file.

Run it

This is a decision model: send typed questions to llama-server's /v1/systemone endpoint, not to chat. Needs llama.cpp release b11572 (12 October 2026, which added d3 support) or newer. Tested at commit fcac190.

llama-server -hf turintech/d3-flash-Artemis-GGUF:IQ4_XS --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

Request example, images and LM Studio

Ollama: no verified supported package is provided here. Use the exact llama.cpp command above; an automatic integration button is not a compatibility test.

llama-server -m d3-flash-Artemis-IQ4_XS.gguf --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

Tested on an Apple M3 Max (36 GB) with llama.cpp Metal (b11617): every file here loads and answers with the command above.

Keep -ub 4096 -b 4096: d3 reads the whole prompt in one batch, so an input longer than the batch size is rejected. -np 1 keeps one request per batch; with several slots, llama.cpp can change a d3 answer when a shorter request shares the batch (ggml-org/llama.cpp#30402, open as of 14 October 2026). -c 16384 stops the server from reserving memory for the full training context.

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Vision inputs

Use ggml-org's vision projector for this model and raise the batch size, since several large images can exceed 4,096 tokens:

hf download ggml-org/vllm-d3-flash-GGUF mmproj-vllm-d3-flash-BF16.gguf --local-dir .
llama-server -m d3-flash-Artemis-IQ4_XS.gguf --port 8080 --mmproj mmproj-vllm-d3-flash-BF16.gguf -c 16384 -ub 8192 -b 8192 -np 1

Send images as data URLs in a top-level images array of the /v1/systemone request.

LM Studio

The model is on the LM Studio Hub as turintech/d3-flash-artemis. That entry loads it as a decision model with the settings above: a 16,384-token context, batch sizes of 4,096 and one parallel session (-np 1). Keep the @iq4_xs label when you download it, because without one LM Studio can pick a different file for your hardware:

lms get turintech/d3-flash-artemis@iq4_xs --gguf
lms load turintech/d3-flash-artemis

Then, with LM Studio's server running (port 1234 by default), send decision requests to /v1/systemone and name the model. The chat window and /v1/chat/completions do not run it as a decision model.

curl http://localhost:1234/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "turintech/d3-flash-artemis",
  "state": "I was charged twice for the same order.",
  "questions": {"angry": {"type": "noul", "instructions": "Is the customer angry?"}}
}'

Tested on 25 October 2026 with LM Studio 0.4.26 (headless llmster 0.0.25-1) and its CPU llama.cpp engine 2.64.0 on Linux x86-64. d3-flash-Artemis-IQ4_XS.gguf was downloaded from Hugging Face, its SHA-256 matched, it loaded through a local copy of this Hub definition and /v1/systemone answered a yes/no and a choice question. That is a compatibility check: the numbers on this card come from llama-server.

Only text input was tested in LM Studio.

Choose this model

• Best for routing, triage and typed checks on text with an 8 GB GPU. Images need more: 8.3 GiB at peak.
• Consider another file when you can spare 6.60 GB and want answers a little closer to the original: ggml-org's Q4_K_M gives the same answer 95.0% of the time.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/d3-flash-Artemis-GGUF:IQ4_XS.
• LM Studio's Hub entry turintech/d3-flash-artemis offers every model file above, without the vision projector.

Without a tag, -hf and hf.co fetch d3-flash-Artemis-IQ3_M.gguf, not the default file, so always add the tag.

How the files compare

"Saved" is the file-size reduction versus the BF16 original. Variant links download the exact GGUF; full filenames are in the commands and checksums. File labels are size classes, explained under "File names, sizes and SHA-256 checksums" at the end of the card.

IQ3_M: Slightly less memory (6.0 GiB). Lower quality: changed 2 of 2,000 answers the original was more than 80% sure of (at most 85% sure).

"Same answer as the original" is how often a file picks the same option as the BF16 original on 2,000 test decisions. "Accuracy change" is the difference in accuracy against the original on the same cases, in points. "Confident answers changed" counts answers the original was more than 80% sure of that the file changed (821 of the 2,000 decisions).

The public Q4_K_M from ggml-org (ggml-org/vllm-d3-flash-GGUF, revision 3d53d13, 6.60 GB) gives the same answer 95.0% of the time, accuracy -0.30, and changed no confident answer. It is slightly closer to the original than our Artemis IQ4_XS file on test decisions and images. Held-out accuracy changes are +0.1 points for Artemis IQ4_XS and -2.8 for public Q4_K_M; benchmark-sample results are in the tables below.

Results

Our default file against the public Q4_K_M and the original, on 2,000 typed decisions from the test split. These decisions were also used to choose the file.

The public Q4_K_M is larger and slightly closer to the original, on test decisions and on images. How it was tested.

Typed-decision accuracy on the card's selection panel, with the measured nearby-size alternatives and BF16 original. These comparisons informed file selection and do not establish performance on unseen inputs. Hardware, runtime and other evaluation panels are listed under How it was tested.

Evaluation details and limitations

How it was tested

Compatibility notes

This is a decision model. Run it on llama-server's /v1/systemone endpoint and keep -ub 4096 -b 4096, as in Run it. The llama.cpp and Ollama buttons under "Use this model" start it as a chat model without those settings, which is not a supported way to run it. The LM Studio button imports the file without those settings; use the Hub entry in LM Studio instead. Ollama 0.40.4 does not load this file (tested 14 October 2026), so use llama-server or LM Studio.

Recommendation notes

Download d3-flash-Artemis-IQ4_XS.gguf (4.87 GB, peak GPU memory 6.1 GiB on an NVIDIA RTX 5090, so it should fit an 8 GB GPU for text) and start it with the settings below, then send requests as shown in Run it. It gives the same answer as the original on 94.5% of 2,000 test decisions and changed none of the 821 answers the original was more than 80% sure of. The public Q4_K_M from ggml-org is slightly closer to the original (95.0%) at 6.60 GB.

These files are per-tensor mixes, and their labels are size classes (see the file names table at the end). Use the exact filename or the :IQ4_XS tag; without a tag, app buttons may select a different file.

Size and quality trade-offs

vllm-sr/d3-flash is 17.9 GB in BF16 and needs 17.0 GiB of GPU memory, so a 24 GB card. The default file here is 4.87 GB, 73% smaller (and 26% smaller than the public Q4_K_M), and needs 6.1 GiB, so it should fit an 8 GB GPU for text; with images it needs 8.3 GiB (both measured on an NVIDIA RTX 5090). It gives the same answer as the original on 94.5% of 2,000 test decisions, keeps its accuracy (72.25% against 71.10%), and changed no answer the original was more than 80% sure of, in text or images.

The separate train-split results are labelled held-out in the tables. Their independence from calibration and variant selection has not been verified for every file.

Test results informed candidate selection, so they do not independently measure performance on new inputs. Read them alongside the separate train-split results and public-file comparisons. Agreement measures whether a compressed model keeps the original's answers, including its mistakes. The typed-decisions accuracy figures compare answers with teacher-generated labels.

Every file was compared with the BF16 original and with ggml-org's Q4_K_M, all at identical llama.cpp settings, on:

• 2,000 typed decisions (questions with a fixed answer type: yes/no, pick one, a score) from the test split of LocalLLaMA/typed-decisions: 400 cases with about five questions each,
• 1,000 further decisions from the dataset's train split, reported as held-out below,
• a 300-item stratified sample of the public Decision Index benchmarks. This is a sample for comparing builds, not a Decision Index score,
• 330 image questions from CV-Bench, RealWorldQA and BLINK, with the same BF16 vision projector for every file,
• option shuffling, the longest inputs and repeat runs.

Measured with llama.cpp fcac190 at the settings above (-np 1, one request at a time; images at -ub 8192 -b 8192), every file on one NVIDIA RTX 5090 32 GB.

KL measures how far a file's probabilities move from the original's. The original itself changes 5.1% of its answers when the options are shuffled. Repeat runs gave identical answers for every file. d3 answers each question of a request on its own, so sending the same questions one per request gave identical answers. The Artemis IQ4_XS file's accuracy gain (+1.15 points, 95% range +0.1 to +2.1) is small and should not be read as an improvement on other tasks.

Memory

Peak GPU memory of llama-server on an NVIDIA RTX 5090: text (-ub 4096 -b 4096) and with the BF16 vision projector (-ub 8192 -b 8192).

Agreement with the original

The agreement numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-fcac190-20261014

One NVIDIA RTX 5090, 14 October 2026.

• Reference: TurinTech's BF16 GGUF conversion of vllm-sr/d3-flash (17,920,698,784 bytes), run on the same GPU and settings
• Panel test: LocalLLaMA/typed-decisions test split, typed decisions; dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel heldout: LocalLLaMA/typed-decisions train split, reported as held-out; dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000; text SHA-256: 46a58d63edfd86e23229c78afe8b72307bb4ca9fb0e8df180cabb3c67ec9dcd5
• Panel decision_index: 300-item stratified sample of the public Decision Index benchmarks (not a Decision Index score); dataset: MMLU, ARC-Easy, ARC-Challenge, WinoGrande, HellaSwag, ANLI, BANKING77, CLINC150, MMLU-Pro, When2Call; split: per source benchmark; cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Panel images: 330 image questions from CV-Bench, RealWorldQA and BLINK; dataset: CV-Bench, RealWorldQA, BLINK; cases: 330; decisions: 330; text SHA-256: 46d96d8d9556847a4fd2aecfa930fc47accc15995b0d1f0d83928dee413cf5b1
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer; all of a case's questions in one request
• llama.cpp build: b11609 (commit fcac190)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 1 -ngl 99 with our decision suite as the client, one request at a time; image questions with -ub 8192 -b 8192 --mmproj mmproj-vllm-d3-flash-BF16.gguf (ggml-org's BF16 projector)
• Hardware: one NVIDIA RTX 5090 32 GB
• Settings: 16,384-token context, batch 4,096, one slot, all layers on the GPU
• Measured by: TurinTech measured every file in these tables, including ggml-org's Q4_K_M, with the same build, settings and GPU
• Selection: the test results informed the choice of files, so they are not an independent test
• Source on this card: the table column "Acc. Δ"; the table rows "Same answer as BF16", "Typed-decision accuracy", "Changed answers where BF16 was over 80% sure (text / images)", "Held-out: same answer as BF16", "Held-out accuracy", "Decision Index sample: same answer as BF16", "Decision Index sample accuracy", "Images: same answer as BF16", "Image accuracy", "KL to BF16 (lower is closer)"
• Comparison files: ggml-org: ggml-org/vllm-d3-flash-GGUF at 3d53d13778bf, vllm-d3-flash-Q4_K_M.gguf, SHA-256 8140de6f2e0939b435ac585ecdec97f4aef1545ff01d072fe7f7e4c88ebaa611

Protocol rtx5090-fcac190-20261014, panel test (LocalLLaMA/typed-decisions test split, typed decisions):

Protocol rtx5090-fcac190-20261014, panel heldout (LocalLLaMA/typed-decisions train split, reported as held-out):

Protocol rtx5090-fcac190-20261014, panel decision_index (300-item stratified sample of the public Decision Index benchmarks (not a Decision Index score)):

Protocol rtx5090-fcac190-20261014, panel images (330 image questions from CV-Bench, RealWorldQA and BLINK):

File names, sizes and SHA-256 checksums

File names

File names end in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick a file by tag. Each file is a per-tensor mix of quant types, so the label is a size class: the smallest standard llama.cpp quant whose typical bits per weight are at least the file's. The general.file_type field inside the files reads Q4_K_M, the base type given to llama-quantize; the table shows the tensor types each file holds.

The earlier names were removed from the repository on 26 October 2026; the files under the names above hold the same bytes.

File checksums

Custom versions

We also make fine-tuned and further optimised versions on request: tuned on specific decisions and data, built for the target hardware (CPU servers, laptops, Apple Silicon, edge devices), or compressed from a private model. Book a call with TurinTech.

About and licence

d3-flash is the 9B decision model of Decision 3.0, made by the vLLM Semantic Router Team (vLLM Semantic Router). The model accepts text (and optionally images) with a set of typed questions, and it returns a probability for every allowed answer in a single pass, instead of generating text. See the original model card.

This is an independent TurinTech conversion, not affiliated with or endorsed by the vLLM Semantic Router Team. The files are standard GGUF and run on stock llama.cpp. For images, use ggml-org's vision projector (see below). Changes from the original: the weights are converted to GGUF and quantized for llama.cpp. See NOTICE.

Apache-2.0 (see LICENSE and NOTICE), the same as vllm-sr/d3-flash, which is built on Qwen/Qwen3.5-9B. The original's NOTICE is reproduced in ours.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.