turintech/Kev-9B-Artemis-GGUF

TurinTech models, built with Artemis

Kev-9B for routing, triage and quick checks on a 12 GB GPU. It answers typed questions (yes or no, pick one, a score) with a probability for each answer, in one pass.

• 4.78 GB download for the default file, Kev-9B-Artemis-IQ3_M.gguf.
• Peak GPU memory 9.1 GiB on an NVIDIA A100 80 GB with the command below; the original needs 20.0 GiB.
• Same answer as the original on 92.2% of 2,000 test decisions, against 93.2% for ggml-org's Q4_K_M (6.36 GB); 0 confident answers changed. These decisions also chose the file.

Run it

This is a decision model: send typed questions to llama-server's /v1/systemone endpoint, not to chat. Needs llama.cpp from 9 October 2026 or newer (tested at commit 1e6f04a).

llama-server -hf turintech/Kev-9B-Artemis-GGUF:IQ3_M --port 8080 -c 16384 -ub 4096 -b 4096

Request example, settings, Ollama and LM Studio

llama-server -m Kev-9B-Artemis-IQ3_M.gguf --port 8080 -c 16384 -ub 4096 -b 4096

Tested on an Apple M3 Max (36 GB) with llama.cpp Metal (b11617): every file here loads and answers with the command above.

Keep -ub 4096 -b 4096: the numbers on this card were measured with them, and the batch size moves the probabilities slightly. In our test on an NVIDIA RTX 5090, Kev-9B-Artemis-IQ3_M.gguf gave 0.730 at the default of 512 and 0.727 at 4096 on an 854-token input. Inputs longer than the batch size are accepted: with llama.cpp 1e6f04a and b11609, a 4,914-token input was answered at both settings. -c 16384 stops the server from reserving memory for the full training context.

For concurrent clients, an optional configuration is -np 8 -c 32768 --cache-ram 0: eight request slots with 4,096 tokens each and the host prompt cache disabled. In our NVIDIA RTX 5090 test with llama.cpp fcac190, this used about 0.6 GiB more GPU memory than the command above. Measure the configuration on the intended workload before choosing it.

llama-server -m Kev-9B-Artemis-IQ3_M.gguf --port 8080 -np 8 -c 32768 -ub 4096 -b 4096 --cache-ram 0

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'

Recommended Ollama package

echo "My Stripe integration keeps failing. Please help ASAP." | ollama run turintech/kev-9b-artemis:4.8gb --ask "Does this message express urgency?"

This verified tag selects Kev-9B-Artemis-IQ3_M.gguf. Ollama 0.40.4 was tested; include "model": "turintech/kev-9b-artemis:4.8gb" in the decision request to port 11434. The explicit size tag selects the recommended file and is not an immutable revision.

Run it with Ollama

Tested with Ollama 0.40.4 on 14 October 2026, on a CPU. Download the file and import it with a 16K context:

hf download turintech/Kev-9B-Artemis-GGUF Kev-9B-Artemis-IQ3_M.gguf --local-dir .
printf 'FROM ./Kev-9B-Artemis-IQ3_M.gguf\nPARAMETER num_ctx 16384\n' > Modelfile
ollama create kev-9b-artemis -f Modelfile
echo "My Stripe integration keeps failing. Please help ASAP." | ollama run kev-9b-artemis --ask "Does this message express urgency?"

The last command prints yes 98%. Ollama serves the same decision endpoint on its own port; add a model field:

curl http://localhost:11434/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "kev-9b-artemis",
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {"urgency": {"type": "noul", "instructions": "Does this message express urgency?"}}
}'

Without num_ctx 16384, Ollama's default context of 4,096 tokens rejects longer inputs. The numbers on this card were measured with llama-server. Ollama sets its own batch size, so the probabilities can differ slightly: 0.978 against 0.977 from llama-server for this example on the same machine.

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

LM Studio

The model is on the LM Studio Hub as turintech/kev-9b-artemis. That entry loads it as a decision model with the settings above: a 16,384-token context and batch sizes of 4,096. Keep the @iq3_m label when you download it, because without one LM Studio can pick a different file for your hardware:

lms get turintech/kev-9b-artemis@iq3_m --gguf
lms load turintech/kev-9b-artemis

Then, with LM Studio's server running (port 1234 by default), send decision requests to /v1/systemone and name the model. The chat window and /v1/chat/completions do not run it as a decision model.

curl http://localhost:1234/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "turintech/kev-9b-artemis",
  "state": "I was charged twice for the same order.",
  "questions": {"angry": {"type": "noul", "instructions": "Is the customer angry?"}}
}'

Tested on 25 October 2026 with LM Studio 0.4.26 (headless llmster 0.0.25-1) and its CPU llama.cpp engine 2.64.0 on Linux x86-64. Kev-9B-Artemis-IQ3_M.gguf was downloaded from Hugging Face, its SHA-256 matched, it loaded through a local copy of this Hub definition and /v1/systemone answered a yes/no and a choice question. That is a compatibility check: the numbers on this card come from llama-server.

Choose this model

• Best for routing, triage and yes or no checks when the public Q4_K_M is too large for your GPU.
• Consider another file when you want the closest match to the original and have the memory: ggml-org's Q8_0 (9.5 GB) gives the same answer 98.7% of the time, and its Q4_K_M (6.36 GB) is also closer.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/Kev-9B-Artemis-GGUF:IQ3_M.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/kev-9b-artemis:<tag>.
• LM Studio's Hub entry turintech/kev-9b-artemis offers every model file above.

How the files compare

"Saved" is the file-size reduction versus the BF16 original. "Acc. Δ" is the accuracy change against the original on the same cases, in percentage points; the other columns are defined below. Variant links download the exact GGUF; full filenames are in the commands and checksums. File labels are size classes, explained under "File names, sizes and SHA-256 checksums" at the end of the card.

Q3_K_S: Tight memory; about 2 to 3.5 points lower accuracy.

"Same answer as the original" is how often a file picks the same option as the BF16 original on 2,000 test decisions. "Confident answers changed" counts answers the original was more than 80% sure of that the file changed.

The usual public files, on the same tests: Q8_0 (ggml-org, 9.5 GB) gives the same answer 98.7% of the time with -0.2 accuracy, and Q4_K_M (ggml-org, 6.36 GB) 93.2% with -0.6. The Q4_K_M is closer to the original than our files (93.2% against 92.2% for the Artemis IQ3_M file; 94.3% against 92.8% on held-out); ours are 1.5 to 2.3 GB smaller and use 1.4 to 2.0 GiB less GPU memory.

Results

Our default file against the closest public file and the original, on 2,000 typed decisions from the test split. These decisions were also used to choose the file.

The public Q4_K_M is larger but stays closer to the original, also on the 1,000 train-split decisions (94.3% against 92.8%). How it was tested.

Typed-decision accuracy on the card's selection panel, with the measured nearby-size alternatives and BF16 original. These comparisons informed file selection and do not establish performance on unseen inputs. Hardware, runtime and other evaluation panels are listed under How it was tested.

Evaluation details and limitations

How it was tested

Compatibility notes

This is a decision model. Run it on llama-server's /v1/systemone endpoint and keep -ub 4096 -b 4096, as in Run it, or import it into Ollama as in Run it with Ollama. The llama.cpp button under "Use this model" starts it as a chat model, which is not a supported way to run it. The LM Studio button imports the file without the context and batch settings above; use the Hub entry in LM Studio instead. The Ollama button runs it as a decision model; add the :IQ3_M tag for the recommended file, because without a tag it can download a smaller one.

Recommendation notes

Download Kev-9B-Artemis-IQ3_M.gguf (4.78 GB, peak GPU memory 9.1 GiB on an NVIDIA A100) and start it with the settings below, then send requests as shown in Run it. It gives the same answer as the original on 92.2% of 2,000 test decisions and changed no answer the original was more than 80% sure of. Go smaller only if memory is tight. The public Q4_K_M from ggml-org is closer to the original (93.2%) at 6.36 GB.

These files are per-tensor mixes, and their labels are size classes (see the file names table at the end). Use the exact filename or the :IQ3_M tag; without a tag, app buttons may select a different file.

Size and quality trade-offs

jaredpalmer/kev-9b is 17.9 GB and needs 20.0 GiB of GPU memory, more than a 16 GB card has. The default file here is 4.78 GB, 73% smaller, and needs 9.1 GiB, so it should fit a 12 GB GPU (both measured on an NVIDIA A100). It gives the same answer as the original on 92.2% of 2,000 test decisions (92.8% on held-out decisions), scores 1.2 accuracy points lower on this sample (67.8% against 69.0%), and changed no answer the original was more than 80% sure of. It is meant for the /v1/systemone decision endpoint.

The separate train-split results are labelled held-out in the tables. Their independence from calibration and variant selection has not been verified for every file.

Test results informed candidate selection, so they do not independently measure performance on new inputs. Read them alongside the separate train-split results and public-file comparisons. Agreement measures whether a compressed model keeps the original's answers, including its mistakes. The typed-decisions accuracy figures compare answers with teacher-generated labels.

Every file was compared with the BF16 original and with the public builds, all at identical llama.cpp settings, on:

• 2,000 typed decisions (questions with a fixed answer type: yes/no, pick one, a score) from the test split of LocalLLaMA/typed-decisions: 400 cases with about five questions each,
• 1,000 further decisions from the dataset's train split, reported separately below (the Artemis IQ3_M default was chosen partly on them; see below),
• a 300-item stratified sample of the public Decision Index benchmarks (MMLU, MMLU-Pro, ARC, WinoGrande, HellaSwag, ANLI, BANKING77, CLINC150, When2Call). This is a sample for comparing builds, not a Decision Index score,
• option shuffling, the longest inputs, many-option questions and repeat runs.

Measured with llama.cpp 1e6f04a at the settings above, every file on one NVIDIA A100 80 GB. Public files: ggml-org/Kev-9B-GGUF (Kev-9B-Q8_0.gguf, Kev-9B-Q4_K_M.gguf). The Artemis IQ3_M file was chosen partly on the held-out decisions, so for that file they are a validation set rather than an untouched one.

The original itself changes 9.8% of its answers when the options are shuffled; the compressed files are close to that. Repeat runs gave identical answers for every file. Small differences in this sample should not be treated as general improvements.

Results on other hardware

Same files on other hardware

The same files on two more machines, text checks only, with llama.cpp 1e6f04a (CUDA) and -c 16384 -ub 4096 -b 4096 --parallel 2. Each file is compared with the original run on the same machine, and the original ran fully on the GPU on both. Peak GPU memory is the server process's own allocation as nvidia-smi reports it.

NVIDIA GB10 (DGX Spark)

The GB10 has 128 GB of memory shared by the CPU and GPU. The original's peak was 19.8 GiB.

Compared with the A100, same answer moves by at most 0.4 points (held-out 0.4) and peak memory is 0.2 to 0.3 GiB lower. Accuracy change moves by up to 0.8 points, most for the Artemis IQ4_XS and Artemis Q3_K_S files (-1.9 and -3.0 here, -1.1 and -2.2 on the A100).

NVIDIA RTX 5090 32 GB

The original's peak was 20.2 GiB.

Compared with the A100, same answer moves by at most 0.3 points (held-out 0.7), accuracy change by at most half a point, and peak memory is 0.1 to 0.2 GiB higher.

Memory

Peak GPU memory of llama-server on an NVIDIA A100 80 GB with the settings above.

Agreement with the original

The agreement numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol kev9b-a100-20261012

NVIDIA A100 80 GB, 12 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a)
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel heldout: typed-decisions train split, held-out slice; dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Panel decision_index: Decision Index stratified sample (MMLU, MMLU-Pro, ARC, WinoGrande, HellaSwag, ANLI, BANKING77, CLINC150, When2Call); dataset: MMLU, MMLU-Pro, ARC, WinoGrande, HellaSwag, ANLI, BANKING77, CLINC150 and When2Call (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer; all of a case's questions in one request
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 with our decision suite as the client
• Hardware: one NVIDIA A100 80 GB per file
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU
• Measured by: TurinTech measured every file in these tables, including ggml-org's Q8_0 and Q4_K_M, with the same build and settings
• Selection: the test results informed the choice of files, so they are not an independent test
• Source on this card: the table column "Acc. Δ"; the table rows "Typed-decision accuracy", "Same answer as BF16", "Held-out accuracy", "Held-out: same answer as BF16", "Decision Index sample accuracy", "Decision Index sample: same answer as BF16", "Changed answers where BF16 was over 80% sure"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF, Kev-9B-Q8_0.gguf, SHA-256 d30b225bfdc985d1856bb04a990be76008d0cf40dccbf78938e3069744bc614c; ggml-org: ggml-org/Kev-9B-GGUF, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol kev9b-a100-20261012, panel test (typed-decisions test split (selection panel)):

Protocol kev9b-a100-20261012, panel heldout (typed-decisions train split, held-out slice):

Protocol kev9b-a100-20261012, panel decision_index (Decision Index stratified sample (MMLU, MMLU-Pro, ARC, WinoGrande, HellaSwag, ANLI, BANKING77, CLINC150, When2Call)):

Protocol kev9b-gb10-20261013

NVIDIA GB10, 13 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a), run on the same machine
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel heldout: typed-decisions train split, held-out slice; dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer; all of a case's questions in one request
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 -t 20 with our decision suite as the client, two workers
• Hardware: one NVIDIA GB10 (128 GB unified memory)
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU, one file at a time, text checks only
• Measured by: TurinTech measured every file in this table, including ggml-org's Q8_0 and Q4_K_M, on the same machine; each file is compared with the original run on that machine
• Selection: the test results informed the choice of files, so they are not an independent test
• Source on this card: the table columns "Same answer as BF16", "Accuracy change", "Confident answers changed", "Held-out: same answer"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q8_0.gguf, SHA-256 d30b225bfdc985d1856bb04a990be76008d0cf40dccbf78938e3069744bc614c; ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol kev9b-gb10-20261013, panel test (typed-decisions test split (selection panel)):

Protocol kev9b-gb10-20261013, panel heldout (typed-decisions train split, held-out slice):

Protocol kev9b-rtx5090-20261013

NVIDIA RTX 5090, 13 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-9B-BF16.gguf (SHA-256 7097686ceab17858c478644e14bee187b5bec775fe4fd6a14aaf02715a903f0a), run on the same kind of GPU
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel heldout: typed-decisions train split, held-out slice; dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer; all of a case's questions in one request
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 --fit off -ngl 99 -t 32 with our decision suite as the client, two workers
• Hardware: NVIDIA RTX 5090 32 GB, one server per GPU
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU, text checks only
• Measured by: TurinTech measured every file in this table, including ggml-org's Q8_0 and Q4_K_M, with the same build and settings; each file is compared with the original run on the same hardware
• Selection: the test results informed the choice of files, so they are not an independent test
• Source on this card: the table columns "Same answer as BF16", "Accuracy change", "Confident answers changed", "Held-out: same answer"
• Comparison files: ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q8_0.gguf, SHA-256 d30b225bfdc985d1856bb04a990be76008d0cf40dccbf78938e3069744bc614c; ggml-org: ggml-org/Kev-9B-GGUF at ec2bbfe66202, Kev-9B-Q4_K_M.gguf, SHA-256 86a6084984a6ef6818eb12c07cdc67ed3c47b00208d41a8816c64301bc6bcbda

Protocol kev9b-rtx5090-20261013, panel test (typed-decisions test split (selection panel)):

Protocol kev9b-rtx5090-20261013, panel heldout (typed-decisions train split, held-out slice):

File names, sizes and SHA-256 checksums

File names

File names end in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick a file by tag. Each file is a per-tensor mix of quant types, so the label is a size class: the smallest standard llama.cpp quant whose typical bits per weight are at least the file's. The general.file_type field inside the files reads Q4_K_M, the base type given to llama-quantize; the table shows the tensor types each file holds.

The earlier names were removed from the repository on 26 October 2026; the files under the names above hold the same bytes.

File checksums

Custom versions

We also make fine-tuned and further optimised versions on request: tuned on specific decisions and data, built for the target hardware (CPU servers, laptops, Apple Silicon, edge devices), or compressed from a private model. Book a call with TurinTech.

About and licence

Kev is a 9B decision model by Jared Palmer. The model accepts text with a set of typed questions, and it returns a probability for every allowed answer in a single pass, instead of generating text. Text only. See the original model card.

This is an independent TurinTech conversion, not affiliated with or endorsed by the Kev authors. The files are standard GGUF and run on stock llama.cpp. Changes from the original: the weights are converted to GGUF and quantized for llama.cpp. See NOTICE.

Apache-2.0 (see LICENSE and NOTICE), the same as jaredpalmer/kev-9b, which is post-trained from Qwen/Qwen3.5-9B-Base.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.