turintech/Kev-0.8B-Artemis-GGUF

TurinTech models, built with Artemis

Kev-0.8B for small routing and triage jobs on your own GPU. It answers typed questions (yes or no, pick one, a score) with a probability for each answer, in one pass.

• 0.620 GB download for the default file, Kev-0.8B-Artemis-Q6_K.gguf.
• Peak GPU memory 5.2 GiB on an NVIDIA A100-SXM4-80GB with the command below.
• Same answer as the original on 95.10% of 1,000 final-set decisions, run once after the choice, against 98.20% for ggml-org's Q8_0 (0.812 GB).

Run it

This is a decision model: send typed questions to llama-server's /v1/systemone endpoint, not to chat. Needs llama.cpp b11540 or newer. Tested at commit 1e6f04a (b11540), built for CUDA, on an NVIDIA A100.

llama-server -hf turintech/Kev-0.8B-Artemis-GGUF:Q6_K --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

Download, request example, settings, Ollama and LM Studio

Or download the exact file and start it:

hf download turintech/Kev-0.8B-Artemis-GGUF Kev-0.8B-Artemis-Q6_K.gguf --local-dir .
llama-server -m Kev-0.8B-Artemis-Q6_K.gguf --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

llama-server -m Kev-0.8B-Artemis-Q6_K.gguf --port 8080 -c 16384 -ub 4096 -b 4096 -np 1

The numbers on this card were measured with these settings and one request slot (-np 1); other slot counts were not tested. -c 16384 stops the server from reserving memory for the full training context. Leave the server running and use a second terminal for the request.

Save this request as request.json. A request has one state and named questions of three types: choice (pick one option), noul (yes or no) and score (a level on a scale):

{
  "state": "I was charged twice for order 1182. Please refund one charge.",
  "questions": {
    "team": {
      "type": "choice",
      "instructions": "Which team should handle this?",
      "criteria": {
        "billing": "Payments and refunds",
        "technical": "Technical faults",
        "sales": "New purchases"
      }
    },
    "refund": {
      "type": "noul",
      "instructions": "The customer requests a refund."
    },
    "urgency": {
      "type": "score",
      "instructions": "How urgent is this request?",
      "criteria": [
        "Low",
        "Medium",
        "High"
      ]
    }
  }
}

Send it:

curl -s http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' -d @request.json | python3 -m json.tool

Output from this file on an NVIDIA A100 with llama.cpp 1e6f04a built for CUDA:

{
    "model": "Kev-0.8B-Artemis-Q6_K.gguf",
    "answers": {
        "team": {
            "type": "choice",
            "choice": "billing",
            "probabilities": {
                "billing": 0.668503409961768,
                "technical": 0.028632709048515014,
                "sales": 0.302863880989717
            },
            "confidence": 0.502755114942652
        },
        "refund": {
            "type": "noul",
            "noul": 0.9234754099979434
        },
        "urgency": {
            "type": "score",
            "score": 0.8803753649420195,
            "legend": {
                "0": "Low",
                "1": "Medium",
                "2": "High"
            },
            "probabilities": {
                "0": 0.3193411048578387,
                "1": 0.4809424253423032,
                "2": 0.19971646979985816
            },
            "confidence": 0.2214136380134547
        }
    },
    "usage": {
        "input_tokens": 112,
        "output_tokens": 0
    }
}

noul is the probability of yes, and score is the expected level.

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Apps and Ollama

The llama.cpp button under "Use this model" starts it as a chat model, which is not a supported way to run it. The LM Studio button imports the file without the context and batch settings above; use the Hub entry in LM Studio instead. Otherwise use llama-server and /v1/systemone as above, or the Ollama package.

Recommended Ollama package

echo "My Stripe integration keeps failing. Please help ASAP." | ollama run turintech/kev-0.8b-artemis:q6_k --ask "Does this message express urgency?"

The q6_k tag selects Kev-0.8B-Artemis-Q6_K.gguf, and so do turintech/kev-0.8b-artemis without a tag and :0.6gb. Tested on 25 October 2026 with Ollama 0.40.4, where the command above prints yes 75%. The package sets a 16K context and a batch size of 4,096 (num_ctx 16384, num_batch 4096). To send request.json to Ollama, add "model": "turintech/kev-0.8b-artemis:q6_k" and use Ollama's port (11434 by default). The numbers on this card were measured with llama-server, so probabilities through Ollama can differ slightly. The explicit tag selects the recommended file and is not an immutable revision.

LM Studio

The model is on the LM Studio Hub as turintech/kev-0.8b-artemis. That entry loads it as a decision model with the settings above: a 16,384-token context and batch sizes of 4,096. Keep the @q6_k label when you download it, because without one LM Studio can pick a different file for your hardware:

lms get turintech/kev-0.8b-artemis@q6_k --gguf
lms load turintech/kev-0.8b-artemis

Then, with LM Studio's server running (port 1234 by default), send decision requests to /v1/systemone and name the model. The chat window and /v1/chat/completions do not run it as a decision model.

curl http://localhost:1234/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "turintech/kev-0.8b-artemis",
  "state": "I was charged twice for the same order.",
  "questions": {"angry": {"type": "noul", "instructions": "Is the customer angry?"}}
}'

Tested on 25 October 2026 with LM Studio 0.4.26 (headless llmster 0.0.25-1) and its CPU llama.cpp engine 2.64.0 on Linux x86-64. Kev-0.8B-Artemis-Q6_K.gguf was downloaded from Hugging Face, its SHA-256 matched, it loaded through a local copy of this Hub definition and /v1/systemone answered a yes/no and a choice question. That is a compatibility check: the numbers on this card come from llama-server. A fresh lms get from the Hub returned the same file (same SHA-256), and it answered too.

The Hub entry does not limit LM Studio to one request slot, and this card's numbers were measured with -np 1.

Choose this model

• Best for small routing and triage jobs where the smallest download matters most.
• Consider another file when you want the closest match to the original: ggml-org's Q8_0 (0.812 GB) gives the same answer 98.20% of the time.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/Kev-0.8B-Artemis-GGUF:Q6_K.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/kev-0.8b-artemis:<tag>.
• LM Studio's Hub entry turintech/kev-0.8b-artemis offers every model file above, without the importance matrix.

Text only; CPU-only memory was not measured.

How the files compare

"Saved" is the file-size reduction versus the BF16 original, whose size here is ggml-org's Kev-0.8B-BF16.gguf, 1.518 GB. The other columns come from the final set. "Acc. Δ" is the accuracy change against the original in percentage points. "Conf. changes" counts answers the original was more than 80% sure of (45 of the 1,000) that the file changed. The variant link downloads the exact GGUF. The label is explained under "File names, sizes and SHA-256 checksums" at the end of the card.

Trade-off: ggml-org's Q8_0 is 31% larger and stays closer to the original: on the final set it kept the original's top answer on 98.20% of decisions against 95.10% for this file (paired difference -3.10 points, 95% interval -4.60 to -1.70). Accuracy does not separate them.

Results

Our file against ggml-org's Q8_0 and the original on the final set: 1,000 decisions from 200 cases that were set aside before any compressed file was scored and run once after the choice.

The Q8_0 is larger and keeps more of the original's answers; accuracy does not separate them. How it was tested.

How often each file kept the original's top answer on the final set: 200 cases (1,000 decisions) reserved before scoring and run once, after this file was chosen. Higher is better; the original is the reference and is not plotted. This file is 24% smaller than ggml-org's Q8_0 and keeps fewer of the original's answers; accuracy does not separate them. See How it compares.

How it compares

How it compares

Typed decisions from LocalLLaMA/typed-decisions, with every file on the same llama.cpp build and settings. The final set is 200 cases (1,000 decisions) reserved from the train split before any compressed file was scored, and run once after this file was chosen. The development set is 200 other cases (1,000 decisions); it chose this file, so its results are exposed to that selection. Accuracy is against the dataset's synthetic teacher labels. "Same answer as original" measures how often the original's top answer is kept, including its mistakes. The dataset is public and may overlap with the original model's training data.

Final set: 200 cases, 1,000 decisions, 45 of them answered by the original with more than 80% probability.

Development set: 200 cases, 1,000 decisions, 58 of them answered by the original with more than 80% probability. These cases chose this file.

These development numbers are a fresh rerun on the same cases, after the file was chosen. Mean KL and same answer are measured against the original's probabilities saved during the search; the fresh run of the original matched them on 997 of 1,000 decisions, and its accuracy is the one shown. Option shuffling was measured on the final set only.

Brackets are 95% percentile intervals from a paired bootstrap over cases (10,000 resamples), so questions sharing a case move together. For accuracy, the intervals against the original and against the Q8_0 include zero, which does not show that the files are equally accurate. Accuracy is below 50% for every file, including the original, so test the model on the target workload before relying on its decisions.

Evaluation details and limitations

How it was tested

Test data and exposure

LocalLLaMA/typed-decisions asks typed questions about a state: pick one option (choice), yes or no (noul), or a level on a scale (score), five questions per case, labelled by a teacher model. From its train split, 200 development cases, 200 final cases and the calibration text were fixed before any compressed file was scored; no case ID or normalised state appears in more than one of them. The development set chose this file from the candidates of the search; the final set ran once, after that choice, on this file, the original, ggml-org's Q8_0 and one other candidate for reference. The choice was not revisited. The dataset is public and may be in the original model's training data.

Runtime and metrics

llama.cpp 1e6f04a (b11540), built with CUDA, on one NVIDIA A100-SXM4-80GB, with -c 16384 -ub 4096 -b 4096 -np 1, all layers on the GPU and 16 CPU threads, sending all of a case's questions in one request. The original is ggml-org's Kev-0.8B-BF16.gguf, and the comparison file is ggml-org's Kev-0.8B-Q8_0.gguf, both from ggml-org/Kev-0.8B-GGUF revision 9231cb7, checked by SHA-256.

• Mean KL to original: for each decision, the sum over options of p_original × ln(p_original / p_file), averaged over decisions.
• Same answer as original: the share of decisions where the file's most probable option is the original's.
• Confident answers changed: decisions where the original's top probability is above 0.8 and the file picks another option.
• Answers changed by option shuffling: 300 choice decisions of the final set asked again with their options in three random orders (900 answers), and the share whose top answer changes.

This card makes no speed claims.

Conversion check

ggml-org's BF16 GGUF, the original on this card, was compared with Jared Palmer's own Kev code (tag kev-1.0, adapter revision bf75a6a8, fp32) on 20 test-split cases (100 decisions), against a gate of at least 98% same top answer and mean KL at most 0.005. It passed: 99% same top answer and mean KL 0.0000056. The one differing decision was a 50/50 yes-or-no answer: the original said no at 0.501, the conversion said yes at 0.502.

Calibration and the file

The file was quantized with llama.cpp llama-quantize (b11540) from that BF16 conversion, using an importance matrix computed with llama-imatrix over state and question text, without labels, from the train split of LocalLLaMA/typed-decisions, excluding the development and final cases. The importance matrix is published in this repository as Kev-0.8B-Artemis-imatrix.gguf_file; it is not a model file. The GGUF header names that file and the dataset in quantize.imatrix.file and quantize.imatrix.dataset, holds no local path, and reads TurinTech in general.quantized_by. The evaluations above ran on an earlier header of this file with byte-identical tensors.

Memory

Peak GPU memory of llama-server on an NVIDIA A100-SXM4-80GB with llama.cpp 1e6f04a, the command in Run it and the example request sent three times: 5.2 GiB, the highest nvidia-smi sample taken every half second. The weights are 0.62 GB of that; the rest is llama.cpp's working memory at these settings. CPU-only memory and the other files' memory were not measured.

Agreement with the original

The agreement numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol kev08b-a100-review-20261024

One NVIDIA A100 80 GB, 24 October 2026.

• Reference: ggml-org's BF16 GGUF Kev-0.8B-BF16.gguf (1,517,798,944 bytes, SHA-256 5bb957b9369e71b44dd6b80546992033f5fbdd128061727f122c7869e2d90de9) of jaredpalmer/kev-0.8b revision bf75a6a8848ea6960ff2ed108d9ed44c2941174f; it matched the original Kev code on a 100-decision conversion check
• Panel final: final set: 200 typed-decisions train-split cases reserved before any compressed file was scored; dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000; text SHA-256: 51141f39b93badf365e470ddd808dcb8fb0f6c7a580d9ca75b0843ecc94b1a9c (SHA-256 of the file holding both the final and the development cases)
• Panel dev: development set: 200 other typed-decisions train-split cases; dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000; text SHA-256: 25d19a5b06c00413da87f82e2970a1e7038dde472a332a76c11bd0ed38e3a476
• Tool: llama-server /v1/systemone, which returns a probability for every allowed answer; all of a case's questions in one request
• llama.cpp build: b11540 (commit 1e6f04a75e91da4bb66b5248f3fd4bdeff169d41)
• Command: llama-server -m <file.gguf> -c 16384 -ub 4096 -b 4096 -np 1 -ngl 99 -t 16 with our decision suite as the client
• Hardware: one NVIDIA A100 80 GB, 16 CPU threads
• Settings: 16,384-token context, batch 4,096, one slot, all layers on the GPU
• Measured by: TurinTech measured every file in these tables, including ggml-org's Q8_0, with the same build, settings and GPU. Our numbers were measured on an earlier header of the published file with the same tensors
• Selection: the final set (200 cases) was reserved from the train split before any compressed file was scored and run once after this file was chosen; the development set was used in the search
• Source on this card: the table columns "Same answer as original", "Accuracy", "Confident answers changed", "Acc. Δ", "Mean KL to original"
• Comparison files: ggml-org: ggml-org/Kev-0.8B-GGUF at 9231cb757e33, Kev-0.8B-Q8_0.gguf, SHA-256 27278f34eb3273bceea4c053dc50dd61a5161da21a718c4aacdf8fd5830771d0

Protocol kev08b-a100-review-20261024, panel final (final set: 200 typed-decisions train-split cases reserved before any compressed file was scored):

Protocol kev08b-a100-review-20261024, panel dev (development set: 200 other typed-decisions train-split cases):

File names, sizes and SHA-256 checksums

File names

The file name ends in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick the file by tag. Q6_K makes up 90.9% of the tensor bytes, so the label is that type. Over the whole file and all 752,917,824 weights it has 6.59 bits per weight. The general.file_type field inside the file also reads Q6_K.

File checksums

About and licence

Kev-0.8B is the smallest Kev decision model by Jared Palmer, a LoRA adapter and pointer head on Qwen3.5-0.8B-Base. The model accepts text with a set of typed questions, and it returns a probability for every allowed answer in a single pass, instead of generating text. Text only. See the original model card.

This is an independent TurinTech conversion, not affiliated with or endorsed by the Kev authors. The file is standard GGUF and runs on stock llama.cpp. It was quantized from ggml-org's BF16 GGUF conversion of Kev-0.8B; that conversion matched the original Kev code on a 100-decision check (see Conversion check). Changes from the original: the weights are converted to GGUF and quantized for llama.cpp. See NOTICE.

Apache-2.0 (see LICENSE and NOTICE), the same as jaredpalmer/kev-0.8b, which is fine-tuned from Qwen/Qwen3.5-0.8B-Base.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.