turintech/Kev-4B-Artemis-GGUF

TurinTech models, built with Artemis

Kev-4B for routing, triage and typed checks on one consumer GPU, compressed with Artemis

Run it

This is a decision model: it answers typed questions (yes or no, pick one, a score) with a probability for each answer, in one pass, on llama-server's /v1/systemone endpoint. It does not chat. Needs llama.cpp b11540 or newer (tested at 1e6f04a).

llama-server -hf turintech/Kev-4B-Artemis-GGUF:Q4_K_M --port 8080 -c 16384 -ub 4096 -b 4096

Ollama: turintech/kev-4b-artemis with --ask, as in Run it  ·  LM Studio: turintech/kev-4b-artemis

Request example, settings, Ollama and LM Studio

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "My Stripe integration keeps failing. Please help ASAP.",
  "questions": {
    "urgency": {"type": "noul", "instructions": "Does this message express urgency?"}
  }
}'

Keep -ub 4096 -b 4096: the numbers on this card were measured with them, and the batch size moves the probabilities slightly (in an RTX 5090 test on a separate 854-token input, the default file gave 0.803 at the default of 512 and 0.815 at 4096). Inputs longer than the batch size are accepted. -c 16384 stops the server from reserving memory for the full training context. For several concurrent clients, -np 8 -c 32768 --cache-ram 0 gives eight slots of 4,096 tokens with the host prompt cache off; in an RTX 5090 test with llama.cpp fcac190 this used about 0.6 GiB more GPU memory than the command above. To run a downloaded file:

hf download turintech/Kev-4B-Artemis-GGUF Kev-4B-Artemis-Q4_K_M.gguf --local-dir .
llama-server -m Kev-4B-Artemis-Q4_K_M.gguf --port 8080 -c 16384 -ub 4096 -b 4096

Ollama (0.40.4 tested) runs the package's tags as decision models with --ask, and serves /v1/systemone on its own port with a model field added; the package sets the card's context and batch sizes. Ollama's own batch handling moves the probabilities a little (0.927 against 0.926 from llama-server on this example):

echo "My Stripe integration keeps failing. Please help ASAP." | ollama run turintech/kev-4b-artemis:q4_k_m --ask "Does this message express urgency?"

LM Studio (0.4.26 tested, CPU engine 2.64.0) loads the Hub entry turintech/kev-4b-artemis as a decision model with the same settings; keep the @q4_k_m label when downloading, then send decision requests to /v1/systemone on port 1234 with the model named. The chat window and /v1/chat/completions do not run it as a decision model. Of the buttons under "Use this model", llama.cpp starts the file as a chat model, which is not supported, and LM Studio imports it without the context and batch settings above; the commands here are the tested paths.

lms get turintech/kev-4b-artemis@q4_k_m --gguf
lms load turintech/kev-4b-artemis

Results

The test decisions were used to choose the published files, so they favour them; bold marks the best displayed value in each column, an observed extremum, not a significance claim. ggml-org's Q8_0 (4.48 GB) keeps the original's answers far more often, and its Q4_K_M (3.03 GB) also stays a little closer than our default on 1,000 train-split decisions (94.0% against 92.7%, with 0.1 points of accuracy lost against 1.7). Every number here has a 95% range on the results page; only TurinTech Q3_K_M's accuracy-change range lies entirely below zero. Full results, memory and method.

Files

Text only. On a CPU alone the default file peaked at 3.78 GiB of RAM over three short requests; over a long run llama-server's default prompt cache (up to 8 GiB) adds to this unless --cache-ram 0 is set. Peak GPU memory measured on NVIDIA RTX 5090, llama.cpp 1e6f04a (b11540), -c 16384 -ub 4096 -b 4096 --parallel 2, all layers on the GPU, the server's peak over the whole 12 October 2026 suite; CPU only: AMD Ryzen 9 5900X, 12 threads, llama.cpp b11617 CPU build, the same context and batch settings with -np 1, peak resident memory over three requests with the Kev-0.8B card's three-question example (112 tokens), the memory-mapped file included, before the default prompt cache fills.

Pick this model

• Good for routing, triage and yes or no checks with typed answers and probabilities on one consumer GPU, in a 2.54 GB download that keeps 93.0% of the original's test answers.
• Pick another file when the closest match to the original matters more: ggml-org's Q8_0 (4.48 GB) gives the same answer on 98.4% of the test decisions, its Q4_K_M (3.03 GB) on 93.2%. When memory is tight, our IQ3_M (2.17 GB) and Q3_K_M (2.04 GB) need 7.2 and 7.1 GiB on the RTX 5090 and keep 90.3% and 88.6% of the original's answers; the Q3_K_M loses 2.2 points of accuracy.

How it was made

Kev-4B is Jared Palmer's 4B decision model (original card), fine-tuned from Qwen3.5-4B-Base; text only. These files were built with Artemis, TurinTech's optimisation platform. Each file is a per-tensor mix of standard llama.cpp quant types, so its label is a size class (the smallest standard quant whose typical bits per weight are at least the file's; the bits per weight and tensor types are on the results page). Stock llama.cpp, Ollama and LM Studio run the files. The weights are converted to GGUF and quantized for llama.cpp; nothing else changes from the original.

How it was tested

Every file in the Results table, the BF16 original included, ran through the same client on one NVIDIA RTX 5090 with llama.cpp b11540 at the context and batch settings above, with two slots (--parallel 2) and two client workers, on 27 October 2026: 2,000 typed decisions from the test split of LocalLLaMA/typed-decisions, a 300-item sample of the public Decision Index benchmarks, option reordering, both request shapes and a repeat run. The test decisions were used to choose the published files, so they favour them. The results page has the method, every table with its 95% ranges, the train-split decisions and the memory of the 12 October run behind the first card, a second run on an NVIDIA GB10, the CPU-only memory, and the files' revisions and checksums.

Agreement with the original

The agreement numbers from the tables on the results page, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol rtx5090-2026-10-27

One NVIDIA RTX 5090, 27 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-4B-BF16.gguf (8,427,020,704 bytes, SHA-256 c690ee43d8244f7cd7dd960aebaadda3666e68bd69aba1af9135737b4f0595d9) of jaredpalmer/kev-4b revision 6cfce5c, from ggml-org/Kev-4B-GGUF revision d924f2e; the same tensors as a fresh conversion with the pinned converter, only general.name differs. Its answer probabilities on the same machine and build are the reference for every file
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel decision_index: Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each); dataset: ANLI, ARC, BANKING77, CLINC150, HellaSwag, MMLU, MMLU-Pro, When2Call and WinoGrande (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points, exact to two decimals. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals (1,000 resamples, seed 20261010)
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41), built with CUDA 13.3
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 -t 16 with our decision suite as the client, two workers
• Hardware: one NVIDIA GeForce RTX 5090 32 GB (AMD EPYC 7V13 host)
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU
• Measured by: TurinTech downloaded every file, ours and ggml-org's, at the revisions below on 27 October 2026, checked each by SHA-256 against the Hub listing, and measured all of them with the same build, settings and GPU
• Selection: the test decisions were used to choose the published files, so they are not an independent test; the 12 October run also measured the Decision Index sample on four candidate files of ours, including one that was not published, and no record says whether it informed the choice
• Source, on the results page: the table columns "Same answer as original, % of 2,000", "KL to original", "Accuracy, %", "Accuracy change, points", "Confident answers changed, of 244", "Same answer as original, %", "Accuracy, % of 300"
• Notes: On these two panels this run reproduced the 12 October run (protocol rtx5090-2026-10-12) to the last digit.
• Comparison files: ggml-org: ggml-org/Kev-4B-GGUF at d924f2e2c387, Kev-4B-Q8_0.gguf, SHA-256 7c2ebed90560522c2801389db482ac1dc4c36d828f201f6074c1d60e433948da; ggml-org: ggml-org/Kev-4B-GGUF at d924f2e2c387, Kev-4B-Q4_K_M.gguf, SHA-256 33ae6b18926502b2209a1bf7d3b61a350d65938441515c686bb87d226a19eff9

Protocol rtx5090-2026-10-27, panel test (typed-decisions test split (selection panel)):

Protocol rtx5090-2026-10-27, panel decision_index (Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each)):

Protocol rtx5090-2026-10-12

One NVIDIA RTX 5090, 12 October 2026 (the run behind the first card).

• Reference: ggml-org's BF16 GGUF Kev-4B-BF16.gguf (8,427,020,704 bytes, SHA-256 c690ee43d8244f7cd7dd960aebaadda3666e68bd69aba1af9135737b4f0595d9) of jaredpalmer/kev-4b revision 6cfce5c, from ggml-org/Kev-4B-GGUF revision d924f2e; the same tensors as a fresh conversion with the pinned converter, only general.name differs. Its answer probabilities on the same machine and build are the reference for every file
• Panel train_split: typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points, exact to two decimals. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals (1,000 resamples, seed 20261010)
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41), built with CUDA 13.3
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 -t 32 with our decision suite as the client, two workers
• Hardware: one NVIDIA GeForce RTX 5090 32 GB, two files at a time on the GPU after the original ran alone
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU
• Measured by: TurinTech measured every file in this table, including ggml-org's Q8_0 and Q4_K_M at revision d924f2e, with the same build, settings and GPU. Our files were the copies before a metadata-only change to their general.name field, with the same tensors as the published files, which reproduced this run's test-decision and Decision Index numbers on 27 October; our records do not hold the copies' revision or SHA-256
• Selection: the train-split decisions were drawn outside the calibration window as this run defined it; their independence from calibration and from the choice of files has not been verified for every file
• Source, on the results page: the table columns "Same answer as original, % of 1,000", "KL to original", "Accuracy, %", "Accuracy change, points", "Confident answers changed"
• Notes: Its test-decision and Decision Index numbers equal the 27 October protocol's and are printed once, under that protocol.
• Comparison files: ggml-org: ggml-org/Kev-4B-GGUF at d924f2e2c387, Kev-4B-Q8_0.gguf, SHA-256 7c2ebed90560522c2801389db482ac1dc4c36d828f201f6074c1d60e433948da; ggml-org: ggml-org/Kev-4B-GGUF at d924f2e2c387, Kev-4B-Q4_K_M.gguf, SHA-256 33ae6b18926502b2209a1bf7d3b61a350d65938441515c686bb87d226a19eff9

Protocol rtx5090-2026-10-12, panel train_split (typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010)):

Protocol gb10-2026-10-13

One NVIDIA GB10, 13 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-4B-BF16.gguf (8,427,020,704 bytes, SHA-256 c690ee43d8244f7cd7dd960aebaadda3666e68bd69aba1af9135737b4f0595d9) of jaredpalmer/kev-4b revision 6cfce5c, from ggml-org/Kev-4B-GGUF revision d924f2e; the same tensors as a fresh conversion with the pinned converter, only general.name differs. Its answer probabilities on the same machine and build are the reference for every file
• Panel test: typed-decisions test split (selection panel); dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 4f294f218ea1da27f3efef936359389c62ea4d3973a41457732990f1d31b647c
• Panel train_split: typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000
• Panel decision_index: Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each); dataset: ANLI, ARC, BANKING77, CLINC150, HellaSwag, MMLU, MMLU-Pro, When2Call and WinoGrande (300-item stratified sample); cases: 300; decisions: 300; text SHA-256: 9cb4301cc7d7d96dde9663263c53020912ad628201e5609ab2d65bf16ebb652e
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer, with all of a case's questions in one request. Same answer: the file's most likely answer is the original's. Mean KL: KL divergence from the original's answer probabilities to the file's, averaged over decisions. Accuracy: against the panel's labels, which are synthetic for typed-decisions and each benchmark's own answer for the Decision Index sample. Accuracy change: the file's accuracy minus the original's on the same decisions, in points, exact to two decimals. Confident answers changed: decisions where the original gave an answer with more than 80% probability and the file chose another. Ranges on the results page are 95% case-bootstrap intervals (1,000 resamples, seed 20261010)
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41), built for CUDA 13.0 (sm_121)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 --parallel 2 -ngl 99 -t 20 with our decision suite as the client, two workers
• Hardware: one NVIDIA GB10 (128 GB of memory shared by the CPU and GPU), one file at a time
• Settings: 16,384-token context, batch 4,096, two slots, all layers on the GPU
• Measured by: TurinTech measured every file in this table, including ggml-org's Q8_0 and Q4_K_M at revision d924f2e, with the same build, settings and machine. Our files were the three files of these sizes as they stood on 13 October; our records do not hold their revision or SHA-256
• Selection: the test decisions were used to choose the published files, so they are not an independent test; the train-split decisions and the Decision Index sample are as described under protocols rtx5090-2026-10-12 and rtx5090-2026-10-27
• Source, on the results page: the table columns "Same answer as original, % of 2,000, GB10", "KL to original, GB10", "Accuracy, %, GB10", "Accuracy change, points, GB10", "Confident answers changed, of 245", "Same answer as original, % of 1,000, GB10", "KL to original, train split, GB10", "Accuracy, %, train split, GB10", "Accuracy change, points, train split, GB10", "Decision Index same answer, %, GB10", "Decision Index accuracy, % of 300, GB10"
• Comparison files: ggml-org: ggml-org/Kev-4B-GGUF at d924f2e2c387, Kev-4B-Q8_0.gguf, SHA-256 7c2ebed90560522c2801389db482ac1dc4c36d828f201f6074c1d60e433948da; ggml-org: ggml-org/Kev-4B-GGUF at d924f2e2c387, Kev-4B-Q4_K_M.gguf, SHA-256 33ae6b18926502b2209a1bf7d3b61a350d65938441515c686bb87d226a19eff9

Protocol gb10-2026-10-13, panel test (typed-decisions test split (selection panel)):

Protocol gb10-2026-10-13, panel train_split (typed-decisions train split, 200 cases drawn outside the calibration window (seed 20261010)):

Protocol gb10-2026-10-13, panel decision_index (Decision Index stratified sample (ANLI, ARC-Challenge, ARC-Easy, BANKING77, CLINC150 with out-of-scope, HellaSwag, MMLU, MMLU-Pro, When2Call, WinoGrande; 30 items each)):

File checksums

---

Artemis is TurinTech's optimisation platform: teams use it to make their own models smaller and faster, or to optimise their code.

All our models · LinkedIn · Questions and requests

_{Apache-2.0, as released by Jared Palmer (licence); see LICENSE and NOTICE. An independent TurinTech release, not affiliated with or endorsed by Jared Palmer. When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.}

These files keep improving.