turintech/decider-2b-Artemis-GGUF

TurinTech models, built with Artemis

Mapika's decider-2b for typed decisions on a small GPU or a CPU. It reads a state and questions, each with its own options, and returns a probability for every option in one pass.

• 1.273 GB download for the default file, decider-2b-Artemis-Q4_K_M.gguf.
• Peak GPU memory 3.97 GiB on an NVIDIA A100-SXM4-80GB with the default 32,768-token context of decide_gguf.py, and 2.95 GiB on a CPU-only build.
• Same answer as the original on 93.65% of 2,000 final-set decisions, scored once, against 89.45% for Mapika's Q4_K_M (1.274 GB).

Run it

This is a decision model, and it runs only through Mapika's decide_gguf.py script on llama-cpp-python. llama-server's /v1/systemone, Ollama and LM Studio do not serve it as a decision model.

pip install "decider-ai==1.5.0" "llama-cpp-python==0.3.36"   # NVIDIA GPU: see Run it for the CUDA build
hf download Mapika/decider-2b-GGUF --revision 0df001b8fe4e5298c5dd45d875d3795fcdaea0ab \
  --include "*.json" --include "*.py" --include "*.jinja" --local-dir decider-2b
hf download turintech/decider-2b-Artemis-GGUF decider-2b-Artemis-Q4_K_M.gguf --local-dir decider-2b
cd decider-2b

from decide_gguf import GGUFDecider

d = GGUFDecider("decider-2b-Artemis-Q4_K_M.gguf")
d.decide("My card was charged twice for the same purchase.",
         [{"question": "Which department should handle this?", "options": ["billing", "technical", "sales"]},
          {"question": "How urgent is this?", "options": ["low", "medium", "high"]}])

Setup details, CUDA build and apps

Tested with Python 3.12, decider-ai 1.5.0 and llama-cpp-python 0.3.36, on an NVIDIA A100 (llama-cpp-python built with CMAKE_ARGS="-DGGML_CUDA=on") and on a CPU-only build. decide_gguf.py is Mapika's script from Mapika/decider-2b-GGUF at revision 0df001b; it reads decider_config.json and the tokenizer from the folder that holds the GGUF file, so keep them together as in the commands at the top. It uses the GPU only when llama-cpp-python was built with one; GGUFDecider(path, n_gpu_layers=0) runs on the CPU.

For an NVIDIA GPU, build llama-cpp-python with CUDA before installing decider-ai, in a fresh environment (this needs a C++ compiler, CMake and the CUDA toolkit with nvcc on PATH), and check that it reports GPU offload:

CMAKE_ARGS="-DGGML_CUDA=on" pip install --no-cache-dir "llama-cpp-python==0.3.36"
pip install "decider-ai==1.5.0"
python -c "import llama_cpp; print(llama_cpp.llama_supports_gpu_offload())"   # True

If llama-cpp-python is already installed as a CPU build, add --force-reinstall --no-deps to the first line. Either way, pip install "decider-ai==1.5.0" also pulls PyTorch (several GB) as a dependency of decider-ai, although decide_gguf.py does not run the model in PyTorch. Tested on Linux only.

Output of the example at the top, printed as JSON, from this file on an NVIDIA A100 with the CUDA build:

[
  {
    "choice": "billing",
    "confidence": 0.9570321152355892,
    "probs": {
      "billing": 0.9570321152355892,
      "technical": 0.017727678930392152,
      "sales": 0.025240205834018734
    }
  },
  {
    "choice": "high",
    "confidence": 0.44983643347377716,
    "probs": {
      "low": 0.1431381964519971,
      "medium": 0.4070253700742258,
      "high": 0.44983643347377716
    }
  }
]

decide() returns, for each question, the most likely option, its probability and the probability of every option. decide_gguf.py reads every question as a choice, at the choice temperature in decider_config.json (2.465), and that is how every number on this card was measured; yes/no and score questions are asked as choices with their options written out. CPU and GPU builds give slightly different probabilities on the same file. In this example the urgency question is a close call: the CPU build answered medium (0.429) over high (0.420), the GPU build high (0.450) over medium (0.407). The billing answer was the same on both (0.958 and 0.957).

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Apps, servers and Ollama

The file holds no decision metadata, so stock llama.cpp, Ollama and LM Studio see a text model. Loaded in llama-server, Ollama or LM Studio it continues prompts, and its generated text is not its answer. Ollama 0.40.11 answered /v1/systemone with HTTP 400 ("not a llama.cpp decision model") for Mapika's Q4_K_M, which has the same header, and LM Studio (headless llmster 0.0.25-1) listed this file as a text model and refused /v1/systemone for it. The llama.cpp and LM Studio buttons under "Use this model" start it as a chat model, which is not a supported way to run it. Mapika's GGUF card says decider-ai 1.6.0 and newer can also load GGUF files through its Decider class, with per-type temperatures; that path was not tested with this file.

Choose this model

• Best for typed decisions through Mapika's decide_gguf.py on a small GPU or a CPU.
• Consider another file when you want the closest match to the original: Mapika's Q8_0 (2.012 GB) gives the same answer 98.55% of the time.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/decider-2b-Artemis-GGUF:Q4_K_M.

Image input needs Mapika's PyTorch weights.

How the files compare

"Saved" is the file-size reduction versus Mapika's BF16 GGUF, 3.776 GB. The other columns come from the final set. "Acc. Δ" is the accuracy change against BF16 in percentage points. "Conf. changes" counts answers BF16 was more than 80% sure of (351 of the 2,000) that the file changed. The label is explained under "File names, sizes and SHA-256 checksums" at the end of the card.

Trade-off: Mapika's Q8_0 (2.012 GB) is 58% larger and stays closer to BF16: 98.55% same answer against 93.65% for this file (paired difference -4.90 points, 95% interval -6.05 to -3.80). Accuracy does not separate them.

Results

Our file against Mapika's own Q4_K_M and the BF16 model on the final set: 2,000 decisions from 400 cases, kept out of calibration and the search and scored once under a release rule fixed beforehand.

Mapika's Q8_0 (2.012 GB) is larger and keeps more of BF16's answers (98.55%). How it was tested.

How often each file kept the BF16 model's top answer on the final set: the 400-case test split (2,000 decisions), kept out of calibration and the search and scored once under a release rule fixed beforehand. Higher is better; BF16 is the reference and is not plotted. At the same size as Mapika's Q4_K_M, this file keeps more of BF16's answers; Mapika's Q8_0 is 58% larger and keeps more still. Accuracy does not separate the files. See How it compares.

How it compares

How it compares

Typed decisions from LocalLLaMA/typed-decisions, every file read the same way: Mapika's decide_gguf.py on llama-cpp-python 0.3.36 built for CUDA, one NVIDIA A100. The final set is all 400 cases of the test split (2,000 decisions), run once after this file was chosen, under the release rule in Test data and exposure. The development set is 200 cases from the train split (1,000 decisions); it chose this file, so its results are exposed to that selection. "BF16" is Mapika's decider-2b-v12-BF16.gguf read through the same script, not the PyTorch model. Accuracy is against the dataset's synthetic teacher labels. "Same answer as BF16" measures how often BF16's top answer is kept, including its mistakes. The dataset is public and may overlap with the original model's training data.

Final set: 400 cases, 2,000 decisions, 351 of them answered by BF16 with more than 80% probability.

Development set: 200 cases, 1,000 decisions, 181 of them answered by BF16 with more than 80% probability. These cases chose this file.

Brackets are 95% percentile intervals from a paired bootstrap over cases (10,000 resamples), so questions sharing a case move together. The accuracy intervals include zero, which does not show that the files are equally accurate. By question type, this file kept BF16's answer on 91.33% of choice, 94.50% of score and 94.83% of yes/no decisions, against 88.67%, 88.75% and 91.17% for Mapika's Q4_K_M.

Evaluation details and limitations

How it was tested

Test data and exposure

LocalLLaMA/typed-decisions asks typed questions about a state: pick one option (choice), yes or no (noul), or a level on a scale (score), five questions per case, labelled by a teacher model. The final set is the whole test split: 400 cases, 600 choice, 800 score and 600 yes/no decisions. The development set is 200 cases of the train split, and the calibration text is 1,000 other train rows, without labels. The development set picked this file and one runner-up from the 40 candidates of the search. The final set was then run once on both, on BF16 and on Mapika's Q4_K_M and Q8_0, under a release rule written down before it ran: release this file if it passed, otherwise the runner-up if it passed, otherwise nothing. So the final set decided whether a file was released, though not which one was preferred. Both passed. The runner-up is the same size and scored within noise of this file (93.80% same answer, 61.45% accuracy). The dataset is public and may be in the original model's training data.

Runtime and metrics

Mapika's decide_gguf.py (sha256 4ffa9859...9143) with decider-ai 1.5.0 and llama-cpp-python 0.3.36 built for CUDA, on one NVIDIA A100-SXM4-80GB, all layers on the GPU, one decide() call per case with all of the case's questions. Questions were turned into decide() questions with decider-ai's render_question: choice options as "name: description", score levels as "index: level", yes/no as "no: ..." and "yes: ...". BF16, Q8_0 and Q4_K_M are Mapika's files from Mapika/decider-2b-GGUF revision 0df001b, checked by SHA-256. BF16 scored against its own saved answers gave 100% agreement and KL 0.

• Mean KL to BF16: for each decision, the sum over options of p_BF16 × ln(p_BF16 / p_file), averaged over decisions.
• Same answer as BF16: the share of decisions where the file's most probable option is BF16's.
• Confident answers changed: decisions where BF16's top probability is above 0.8 and the file picks another option.

Apart from Mapika's own files, no public GGUF of decider-2b v12 existed when this was tested; the other public GGUFs of Mapika's decider-2b quantize version 10. This card makes no speed claims.

Calibration and the file

The file was quantized with llama.cpp llama-quantize (b11617) from Mapika's BF16 GGUF, using an importance matrix computed with llama-imatrix over state and question text from the train split, one question per record, without labels or development cases. The tensor types were chosen by an Artemis search against the development set. The GGUF header reads decider-2b Artemis Q4_K_M in general.name and holds no local path. The evaluations above ran on the same file with Mapika's original header name; all tensors are byte-identical, and the final set gave identical answers on both.

Memory

Peak GPU memory for the example at the top on an NVIDIA A100-SXM4-80GB with the CUDA build: 3.97 GiB, the highest nvidia-smi sample of the GPU taken every half second, with decide_gguf.py's default context of 32,768 tokens. The weights are 1.27 GB of that. The Python process used 2.05 to 2.40 GiB of system memory alongside the GPU (two runs). Most of the GPU figure is the context and compute buffers: GGUFDecider(path, n_ctx=4096) reserves less, which we did not measure. On the CPU-only build (n_gpu_layers=0) the peak resident memory of the process was 2.95 GiB.

Agreement with the original

The agreement numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Same answer as BF16: the share of decisions where the file picks the same answer as the BF16 original, mistakes included. Higher is closer.
• Mean KL to BF16: KL divergence between the original's and the file's answer probabilities, as the protocol describes. Lower is closer.
• Accuracy: the share of decisions that match the panel's labels. Accuracy change: the file's accuracy minus the original's, in points.
• Confident answers changed: answers the original gave with more than 80% probability that the file changed. Lower is closer.

n/a means the card's tables or our run records do not have that value.

Protocol a100-decide-gguf-20261025

One NVIDIA A100 80 GB, 25 October 2026.

• Reference: Mapika's BF16 GGUF decider-2b-v12-BF16.gguf (3,775,708,704 bytes, SHA-256 4b5682b1d2b94da5b88692862824e9c3cefae4ac0c83a20bfe40762173e41d7d) from Mapika/decider-2b-GGUF at 0df001b8fe4e, read through the same script; base model Mapika/decider-2b revision 7614a6c7ea5a047aa7d7bcfb1a0015469381c3e3
• Panel test: Final set: all of the LocalLLaMA/typed-decisions test split; dataset: LocalLLaMA/typed-decisions; split: test; cases: 400; decisions: 2,000; text SHA-256: 91b97b4d94a17e8f3dc3916793d1fb33c9c5e18361e06673fd14efd3f569b06d
• Panel dev: Development set: 200 cases of the LocalLLaMA/typed-decisions train split, stratified by workflow (seed 20261025); dataset: LocalLLaMA/typed-decisions; split: train; cases: 200; decisions: 1,000; text SHA-256: 98f5c411be098d004d58550c95a0adc357e63f867cd1c6a9eca8d456e8f1b02b
• Tool: Mapika's decide_gguf.py (from Mapika/decider-2b-GGUF at 0df001b8) with decider-ai 1.5.0 on llama-cpp-python 0.3.36 built for CUDA, not llama-server; one call per case with all of the case's questions in one prompt. Mean KL to BF16 is KL(BF16 || file) over each decision's options, averaged over decisions
• llama.cpp build: llama-cpp-python 0.3.36
• Command: our frozen scoring script calling GGUFDecider(model, folder=<Mapika support files>).decide(...) once per case, one process, files in a fixed order with the SHA-256 checked before and after each; the full command line is not public
• Hardware: one NVIDIA A100 80 GB, all layers on the GPU
• Settings: decide_gguf.py defaults (32,768-token context), sequential, one case at a time
• Measured by: TurinTech measured every file in these tables, including Mapika's Q8_0 and Q4_K_M, the same way on the same GPU
• Selection: the final set was kept out of calibration and the search and scored once under a release rule fixed beforehand; the development set was used in the search
• Source on this card: the table columns "Same answer as BF16", "Accuracy", "Confident answers changed", "Acc. Δ", "Mean KL to BF16"
• Comparison files: Mapika: Mapika/decider-2b-GGUF at 0df001b8fe4e, decider-2b-v12-Q8_0.gguf, SHA-256 b89b9bed14dc889075de93136e9a9e3cd859c26b008bfb4bb0211ca0d7357c27; Mapika: Mapika/decider-2b-GGUF at 0df001b8fe4e, decider-2b-v12-Q4_K_M.gguf, SHA-256 e6d3a7534a3abc3a723f8263dc694b86f533214a5bf761474113c66d44018340

Protocol a100-decide-gguf-20261025, panel test (Final set: all of the LocalLLaMA/typed-decisions test split):

Protocol a100-decide-gguf-20261025, panel dev (Development set: 200 cases of the LocalLLaMA/typed-decisions train split, stratified by workflow (seed 20261025)):

File names, sizes and SHA-256 checksums

File names

The file name ends in a standard llama.cpp quant label, so apps can pick the file by tag. No single tensor type makes up 90% of the tensor bytes (Q5_K 52%, Q4_K 27%, Q6_K 21%, F32 0.2%), so the label is a size class: over the whole file and all 1,881,825,088 weights it has 5.41 bits per weight, against 5.42 for Mapika's stock Q4_K_M of the same BF16 file. The general.file_type field inside the file also reads Q4_K_M.

File checksums

About and licence

decider-2b is a decision model by Mapika, fine-tuned from Qwen3.5-2B-Base. It reads a state and one or more questions, each with its own list of options, and returns a probability for every option from one forward pass, instead of generating text. This file is the text model of version 12; image input needs Mapika's PyTorch weights. See the original model card and Mapika's GGUF card.

This is an independent TurinTech quantization, not affiliated with or endorsed by Mapika. It was quantized from Mapika's own BF16 GGUF (decider-2b-v12-BF16.gguf), and it uses Mapika's tokenizer, temperatures and decide_gguf.py unchanged; this repository holds only the GGUF file. Compressed with Artemis Adaptive Compression by TurinTech. Changes from the original: the weights are quantized for llama.cpp. See NOTICE.

Apache-2.0 (see LICENSE and NOTICE), the same as Mapika/decider-2b, which is fine-tuned from Qwen/Qwen3.5-2B-Base. See the original card for its training sources and their terms.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.