turintech/d3-mini-Artemis-GGUF

TurinTech models, built with Artemis

d3-mini for typed decisions on an 8 GB GPU or a CPU. It reads a state and typed questions and returns a probability for each answer, in one pass.

• 3.30 GB download for the default file, d3-mini-Artemis-Q5_K_S.gguf.
• Peak GPU memory 4.46 GiB on an NVIDIA A100-SXM4-80GB and 4.10 GiB on an NVIDIA RTX 4060 8 GB; 4.38 GiB of RAM on an AMD EPYC 7742 CPU.
• Mean KL to the original 0.00349 on 800 final-set decisions, scored once, against 0.00560 for ggml-org's Q4_K_M (3.71 GB).

Run it

This is a decision model: send typed questions to llama-server's /v1/systemone endpoint, not to chat. Needs llama.cpp release b11572 (12 October 2026, which added d3 support) or newer. Tested with v0.7.0 (b11669, commit 620d222).

llama-server -hf turintech/d3-mini-Artemis-GGUF:Q5_K_S --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

Request example, settings, Ollama and LM Studio

llama-server -m d3-mini-Artemis-Q5_K_S.gguf --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

Keep -ub 4096 -b 4096: d3 reads the whole prompt in one batch, so an input longer than the batch size is rejected. -np 1 keeps one request per batch; with several slots, llama.cpp can change a d3 answer when a shorter request shares the batch (ggml-org/llama.cpp#30402, open as of 14 October 2026). -c 16384 stops the server from reserving memory for the full training context.

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My card was charged twice for the same order and nobody has answered my emails for a week. I want this fixed today.",
  "questions": {
    "angry": {"type": "noul", "instructions": "Is the customer angry?"},
    "topic": {"type": "choice", "instructions": "What is the message about?",
              "criteria": {"billing": "payments, charges or refunds", "shipping": "delivery or tracking", "other": "anything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is this request?",
                "criteria": ["Low: can wait.", "Medium: handle this week.", "High: handle today."]}
  }
}'

Answers from this file on an NVIDIA A100 with the command above (probabilities rounded here):

{
  "angry": {"type": "noul", "noul": 0.979},
  "topic": {"type": "choice", "choice": "billing",
            "probabilities": {"billing": 0.993, "shipping": 0.001, "other": 0.007}},
  "urgency": {"type": "score", "score": 1.97,
              "probabilities": {"0": 0.002, "1": 0.023, "2": 0.975}}
}

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Ollama

ollama pull turintech/d3-mini-artemis:q5_k_s
curl http://localhost:11434/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "turintech/d3-mini-artemis:q5_k_s",
  "state": "My card was charged twice for the same order and nobody has answered my emails for a week. I want this fixed today.",
  "questions": {
    "angry": {"type": "noul", "instructions": "Is the customer angry?"},
    "topic": {"type": "choice", "instructions": "What is the message about?",
              "criteria": {"billing": "payments, charges or refunds", "shipping": "delivery or tracking", "other": "anything else"}},
    "urgency": {"type": "score", "instructions": "How urgent is this request?",
                "criteria": ["Low: can wait.", "Medium: handle this week.", "High: handle today."]}
  }
}'

The q5_k_s tag selects d3-mini-Artemis-Q5_K_S.gguf (same SHA-256 as here), and so do :3.3gb and turintech/d3-mini-artemis without a tag. Ollama's /v1/systemone takes the same body as llama-server's plus a model field. The package sets num_ctx 16384 and num_batch 4096. Both are needed: without num_batch 4096, the three long test requests (about 3,300 tokens each) failed with HTTP 400 and "the prompt is longer than the context length currently available to the model", although they fit the context.

Tested on 26 October 2026 with Ollama 0.40.11 on an NVIDIA A100 and on CPU: this request and the three longest development cases (18 questions) gave the same top answers as llama-server, with probabilities within 0.033 of it. Ollama 0.40.4 does not load the file ("unsupported decision model type: vllm-d3"). The numbers on this card were measured with llama-server. The explicit tag selects the recommended file and is not an immutable revision.

LM Studio

The model is on the LM Studio Hub as turintech/d3-mini-artemis. That entry loads it as a decision model with the settings above: a 16,384-token context, batch sizes of 4,096 and one parallel session. Keep the @q5_k_s label when you download it:

lms get turintech/d3-mini-artemis@q5_k_s --gguf
lms load turintech/d3-mini-artemis

Then, with LM Studio's server running (port 1234 by default), send the request above to /v1/systemone with "model": "turintech/d3-mini-artemis". The chat window and /v1/chat/completions do not run it as a decision model.

Tested on 26 October 2026 with LM Studio 0.4.26 (headless llmster 0.0.25-1, lms CLI commit dcefd5a) and its CPU llama.cpp engine 2.64.0 on Linux x86-64, loaded through a local copy of this definition: the request above gave the same top answers as llama-server. Only text input was tested. That is a compatibility check: the numbers on this card come from llama-server.

The llama.cpp button under "Use this model" starts it as a chat model, and the Ollama and LM Studio buttons import the file without the context and batch settings above. Use the package or the Hub entry instead.

Choose this model

• Best for typed decisions on an 8 GB GPU or a CPU.
• Consider another file when you want the closest match to the original: ggml-org's Q8_0 (5.16 GB) gives the same answer 99.50% of the time.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/d3-mini-Artemis-GGUF:Q5_K_S.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/d3-mini-artemis:<tag>.
• LM Studio's Hub entry turintech/d3-mini-artemis offers every model file above.

Text only here.

How the files compare

"Saved" is the file-size reduction versus the BF16 GGUF, 9.70 GB. The other columns come from the final set: 800 held-out decisions, kept out of calibration and the search. "Same answer" is how often the file picks the same option as BF16. "Acc. Δ" is the accuracy change against BF16 in percentage points, measured against the dataset's synthetic teacher labels. "Conf. changes" counts answers BF16 was more than 80% sure of (309 of the 800) that the file changed. The label is explained under "File names, sizes and SHA-256 checksums" at the end of the card.

Trade-off: ggml-org's Q8_0 (5.16 GB) is 57% larger and stays much closer to BF16: 99.50% same answer and mean KL 0.00025, against 97.25% and 0.00349 for this file. Accuracy was 72.50% for the Q8_0 and 72.00% for this file.

Results

Our file against ggml-org's Q4_K_M and the original on the final set: 800 decisions kept out of calibration and the search, and scored once.

ggml-org's Q8_0 (5.16 GB) is larger and stays much closer to BF16, with 99.50% same answer. How it was tested.

How far each file's probabilities move from the BF16 model's on the final set: 160 train-split cases (800 decisions), kept out of calibration and the search and scored once under a release rule fixed beforehand. Lower is closer; BF16 is the reference and is not plotted. This file is smaller than ggml-org's Q4_K_M and closer to BF16; ggml-org's Q8_0 is 57% larger and much closer. Agentware's file was made from an earlier revision of the original. See How it compares.

How it compares

How it compares

Typed decisions from LocalLLaMA/typed-decisions, every file served the same way: llama.cpp v0.7.0 llama-server with -c 16384 -ub 4096 -b 4096 -np 1, all layers on one NVIDIA A100, one request per case with all of its questions. The final set is 160 cases (800 decisions) from the train split, run once after the development set had picked two finalists, under the release rule in Test data and exposure; it decided which of them, if either, was released. The development set is 240 other train-split cases (1,200 decisions); it picked the finalists, so its results are exposed to that selection. "BF16" is our llama.cpp conversion of the original's text and decision weights; ggml-org's BF16 GGUF gave the same answers to every request. "Answers changed by shuffling options" re-asks every choice question with its options in three seeded random orders and counts how often the file's top option moves from its own unshuffled answer. Accuracy is against the dataset's synthetic teacher labels. "Same answer as BF16" measures how often BF16's top answer is kept, including its mistakes.

Final set: 160 cases, 800 decisions (240 choice, 320 score, 240 yes/no), 309 of them answered by BF16 with more than 80% probability, and 720 shuffled choice answers.

Development set: 240 cases, 1,200 decisions, 495 of them answered by BF16 with more than 80% probability, and 1,080 shuffled choice answers. These cases picked the finalists.

Brackets are percentile intervals from a paired bootstrap over cases (10,000 resamples), so questions sharing a case move together: 97.5% for the first three rows, the measures the release rule tested for two candidate files, and 95% for accuracy. Only the KL interval excludes zero. The other intervals include zero, which does not show that the files are equal on those measures.

Agentware's mini-MLP-NVFP4-attention-Q8_0-CUDA-F32readout.gguf (Agentware/d3-mini-calibrated-gguf, revision 67abc2e) ("mini_mixed" is Agentware's name for it) was made from an earlier revision of d3-mini (6dfc4cc), so its distance from this BF16 also includes the change between the two revisions.

Evaluation details and limitations

How it was tested

Test data and exposure

LocalLLaMA/typed-decisions asks typed questions about a state: pick one option (choice), yes or no (noul), or a level on a scale (score), five questions per case, labelled by a teacher model. From its train split, 1,200 cases were split once, before any compressed file was scored, into a 240-case development set, a 160-case final set and 800 calibration rows, balanced across the four workflows. The three are disjoint by case ID. Four final cases share a state template with a calibration row (one is a near copy); the calibration text has no labels, and leaving those four cases out does not change the release decision (the KL interval stays above zero). The development set scored the 40 candidates of an Artemis search and picked two finalists: a primary, the lowest score on the search's objective, and this file as the runner-up, chosen because it was closer to BF16 than ggml-org's Q4_K_M on the development set. The final set was then run once on both, on BF16, on ggml-org's Q4_K_M and Q8_0 and on Agentware's file, under a release rule written down before it ran: release the primary if it passed, otherwise the runner-up if it passed, otherwise nothing. So the final set decided whether a file was released and which one. To pass, a file had to be smaller than ggml-org's Q4_K_M, change no more answers under shuffling than it, be closer to BF16 than it on agreement, KL or shuffling with a 97.5% interval above zero, lose no accuracy significantly against BF16 or the Q4_K_M, and pass nine fixed quality gates. The primary failed the closeness test: its agreement and shuffle differences against ggml-org's Q4_K_M were not significant, and the Q4_K_M was significantly closer to BF16 than it on KL. This file passed on KL. The test split of the dataset was not used. The dataset is public and may be in the original model's training data.

Runtime and metrics

llama.cpp v0.7.0 (620d222) llama-server built for CUDA, on one NVIDIA A100-SXM4-80GB, with -c 16384 -ub 4096 -b 4096 -np 1 -ngl 99. BF16 scored against its own saved answers gave 100% agreement, KL 0 and the same shuffle results, and a second run of 50 cases gave identical answers for every file. Asking each question in its own request gave the same answers as asking all of a case's questions together, for every file. ggml-org's files are from ggml-org/vllm-d3-mini-GGUF revision e7230e2 and were checked by SHA-256.

• Mean KL to BF16: for each decision, the sum over options of p_BF16 × ln(p_BF16 / p_file), averaged over decisions.
• Same answer as BF16: the share of decisions where the file's most probable option is BF16's.
• Confident answers changed: decisions where BF16's top probability is above 0.8 and the file picks another option.

Calibration and the file

The file was quantized with llama.cpp llama-quantize (v0.7.0) from our BF16 conversion of vllm-sr/d3-mini at revision bd0090d, using an importance matrix computed with llama-imatrix over state and question text from the 800 calibration rows, without labels. The tensor types were chosen by an Artemis search against the development set. The GGUF header reads d3-mini Artemis Q5_K_S in general.name. The evaluations above ran on the same file with the header name d3-mini; all tensors are byte-identical, and the final set gave identical answers on both.

Memory and speed

llama-server v0.7.0 with the settings in Run it, 40 development cases sent one at a time (about 1,170 prompt tokens each), after one warm-up pass; the mean of two timed passes. GPU memory is the highest nvidia-smi reading for the server process, sampled every half second; system memory is the server's peak resident memory. CPU runs used 16 threads, pinned to cores 48-63 on the EPYC and 0-15 on the Ryzen. The Ryzen figures are a repeat on an otherwise idle machine: a first run, taken while the machine was busy, had uneven passes and put this file slower than the Q4_K_M.

It fits an 8 GB GPU: on the RTX 4060 it peaked at 4.10 GiB. Against ggml-org's Q4_K_M, this file used 0.38 GiB less GPU memory on both GPUs and about 1.15 GiB less RAM on both CPUs. Speed depends on the hardware: it took 3% longer per request on the A100 and 6% longer on the EPYC, about the same time on the RTX 4060 (386 against 387 ms) and 13% less on the Ryzen. These runs used the same tensors as the release file under an earlier header name. No Intel or ARM CPU was measured.

Agreement with the original

The agreement numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol a100-620d222-20261025

One NVIDIA A100-SXM4-80GB, 25 October 2026.

• Reference: TurinTech's llama.cpp v0.7.0 BF16 GGUF conversion of the text and decision weights of vllm-sr/d3-mini revision bd0090dd758d32ba59ef5c749910a8e87fc4391f (9,695,793,632 bytes, SHA-256 c3662bb121efec2e463b247ce995a4cdf2777aef66d7eafa68f8b4ea995bceee), run on the same GPU and settings; ggml-org's BF16 GGUF gave the same answers to every request
• Panel final: final set: 160 typed-decisions train-split cases, kept out of calibration and the search; dataset: LocalLLaMA/typed-decisions; split: train; cases: 160; decisions: 800; text SHA-256: 387a668bbcdc9b400844b25daec93927c7a768e6aa06b50abd2ced01ededb8e3
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer; all of a case's questions in one request, scored by a frozen scorer whose SHA-256 was checked before the run
• llama.cpp build: v0.7.0, b11669 (commit 620d2225cc436ab0b001e3200e590e3b286ef9ab), built for CUDA
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 1 -ngl 99 -t 4 with our decision suite as the client, one request at a time
• Hardware: one NVIDIA A100-SXM4-80GB
• Settings: 16,384-token context, batch 4,096, one slot, all layers on the GPU
• Measured by: TurinTech measured every file in these tables, including ggml-org's Q8_0 and Q4_K_M and Agentware's file, with the same build, settings and GPU, and checked each file's SHA-256 before and after its run. Our numbers were measured on an earlier header of the published file with the same tensors and gave identical answers
• Selection: the development set picked two finalists; the final set (160 cases) was split off before any compressed file was scored and run once under a release rule fixed beforehand, which chose this file over the other finalist
• Source on this card: the table columns "Same answer as BF16", "Mean KL to BF16", "Accuracy", "Confident answers changed", "Acc. Δ"
• Notes: Agentware's file was made from an earlier revision of d3-mini (6dfc4cc), so its distance from this BF16 also includes the change between the two revisions. Option-shuffle stability and the development set are in the card's own tables, not in this standard table.
• Comparison files: ggml-org: ggml-org/vllm-d3-mini-GGUF at e7230e2309c1, vllm-d3-mini-Q4_K_M.gguf, SHA-256 ab3d4378c268f7f3238a63607af40c0cb516e4482278b01c2f48e9d0698ce522; ggml-org: ggml-org/vllm-d3-mini-GGUF at e7230e2309c1, vllm-d3-mini-Q8_0.gguf, SHA-256 f9f99be46b7ce1e5a5f09ed620de89106447646b34f7d2993e7563b48feb2858; Agentware: Agentware/d3-mini-calibrated-gguf at 67abc2e3b2fb, mini-MLP-NVFP4-attention-Q8_0-CUDA-F32readout.gguf, SHA-256 a1541ef1e0f672964783e8ad177b83917c49eff1a321e255d5a31b0c00e6c00d

Protocol a100-620d222-20261025, panel final (final set: 160 typed-decisions train-split cases, kept out of calibration and the search):

File names, sizes and SHA-256 checksums

File names

The file name ends in a standard llama.cpp quant label, so apps can pick the file by tag. No single tensor type makes up 90% of the tensor bytes (Q5_K 61%, Q4_K 24%, Q8_0 8%, Q6_K 7%), so the label is a size class: the smallest label in a stock ladder we measured (Q3_K_L, IQ4_XS, Q4_K_S, Q4_K_M, Q5_K_S) whose bits per weight are at least the file's. Over the whole file and all 4,841,450,496 weights it has 5.45 bits per weight. Stock llama-quantize files made from the same BF16 with llama.cpp v0.7.0 have 5.07 for Q4_K_M (3.07 GB) and 5.66 for Q5_K_S (3.43 GB), so the class is Q5_K_S. ggml-org's Q4_K_M is larger than both (3.71 GB, 6.13 bits per weight) because it keeps the output matrix and the attention and SSM projections at Q8_0. The general.file_type field inside the file reads Q4_K_M, the base type given to llama-quantize.

File checksums

About and licence

d3-mini is the smallest decision model of Decision 3.0, made by the vLLM Semantic Router Team (vLLM Semantic Router) on Qwen3.5-4B. It reads a state and a set of typed questions and returns a probability for every allowed answer in a single pass, instead of generating text. See the original model card.

This is an independent TurinTech conversion, not affiliated with or endorsed by the vLLM Semantic Router Team. The file is standard GGUF and runs on stock llama.cpp. It holds the text and decision weights only: the original also reads images and video, but no vision projector is included here and image input was not tested with this file. Changes from the original: the weights are converted to GGUF and quantized for llama.cpp. See NOTICE.

Apache-2.0 (see LICENSE and NOTICE), the same as vllm-sr/d3-mini, which is built on Qwen/Qwen3.5-4B. The original's NOTICE is reproduced in ours.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.