GGUF conversions and quantizations of perplexity-ai/pplx-decider-flash-v1.3,
the 5B decision model (Qwen3.5-4B backbone, text and images) that answers noul / choice / score questions in one forward pass
without generating tokens. Every file was checked against Perplexity's reference implementation and benchmarked on public decision suites.
Unofficial. Not made or endorsed by Perplexity. Stock llama.cpp does not run these files correctly: use the pplx-decider-v4-macaq branch of llama.cpp (see below).
| File | Size | How it was made | Use |
|---|---|---|---|
pplx-decider-flash-v1.3-BF16.gguf |
9.70 GB | converted, no quantization | source for your own quantizations |
pplx-decider-flash-v1.3-Q8_0.gguf |
5.16 GB | llama-quantize Q8_0 |
recommended: closest to the reference, among the fastest |
pplx-decider-flash-v1.3-Q6_K.gguf |
3.99 GB | llama-quantize Q6_K, imatrix (general decision data) |
as accurate as Q8_0, 10% slower, 0.8 GB less VRAM |
pplx-decider-flash-v1.3-MACaQ-IQ4_XS.gguf |
2.79 GB | MACaQ IQ4_XS, general decision data | the closest 4-bit file; needs the MACaQ runtime (in the branch) |
pplx-decider-flash-v1.3-MACaQ-IQ2_XS.gguf |
1.93 GB | MACaQ IQ2_XS, general decision data | not recommended: loses about 7 points on text, 4 on vision |
mmproj-pplx-decider-flash-v1.3-BF16.gguf |
0.68 GB | vision tower (65,536 to 1,638,400 pixels) | needed for images, with any of the files |
llama.cpp-pplx-decider-v4-macaq.patch |
28 KB | the changes of the llama.cpp branch as one patch |
Stock llama.cpp has an older pplx-decider decision type (v1 checkpoints, causal, an answer prompt). Decider v1.3 uses protocol v4:
non-causal attention in the full-attention layers, one sequence per question, the answer read at a <|fim_suffix|> marker after a
<|fim_middle|> separator. The pplx-decider-v4-macaq branch of llama.cpp adds:
pplx-decider-v4 in llama-server (/v1/systemone), with the converter class for v1.3 checkpoints;pre_scale vectors and the block Hadamard of macaq.hadamard_block, on llama.cpp's existing
Hadamard-folded weight path (needed by the two MACaQ files only);The branch is llama.cpp b12359 (commit b1b770d) plus 4 commits:
git clone -b pplx-decider-v4-macaq https://github.com/vblanco20-1/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release --target llama-server
llama-server -m pplx-decider-flash-v1.3-Q8_0.gguf --mmproj mmproj-pplx-decider-flash-v1.3-BF16.gguf \
-ngl 99 -np 8 -c 32768 -b 4096 -ub 4096
The same changes as one patch, for a llama.cpp checkout at b1b770d: git apply llama.cpp-pplx-decider-v4-macaq.patch.
A question is read in one batch (non-causal attention), so -b / -ub must hold the longest prompt (state, images and question);
the context per slot is -c / -np. For long documents, use fewer slots and larger batches (-np 2 -c 65536 -b 32768 -ub 32768).
curl http://127.0.0.1:8080/v1/systemone -H "Content-Type: application/json" -d '{
"state": "",
"images": ["data:image/png;base64,..."],
"questions": {
"material": {"type": "choice", "instructions": "Which material does this texture show?",
"criteria": {"wood": "Wood", "stone": "Stone", "metal": "Metal", "other": "Something else"}},
"pixelated": {"type": "noul", "instructions": "Is this image pixelated?"}
}
}'
The prompt follows protocol.py of the base repo exactly (separate mode), including its escaping of special-token text in the
input. A null state is read as the text null, as in the reference; send "" for image-only questions.
11 requests (10 images at 256 to 512 px with 7 questions each, and one text request covering JSON state, special-token text, default instructions and noul criteria), 74 answers, compared with Perplexity's pplx_decider package (Transformers, BF16, CUDA). With the BF16 GGUF every prompt has exactly the reference's token count (images included); the different picks are exact ties in the reference.
| File | Same pick | Median prob. gap | Mean gap | Answers off by > 0.05 | Largest gap |
|---|---|---|---|---|---|
| BF16 | 72/74 | 0.0035 | 0.0064 | 0 | 0.037 |
| Q8_0 | 71/74 | 0.0047 | 0.0081 | 0 | 0.044 |
| Q6_K | 73/74 | 0.0065 | 0.0111 | 1 | 0.069 |
| MACaQ-IQ4_XS | 67/74 | 0.0151 | 0.0243 | 8 | 0.192 |
| IQ4_XS (plain, not in this repo) | 70/74 | 0.0209 | 0.0277 | 10 | 0.150 |
| MACaQ-IQ2_XS | 68/74 | 0.0582 | 0.0778 | 40 | 0.395 |
Accuracy only; random samples, the same rows for every model (seeds below). The reference is Perplexity's own implementation; three Liquid d1 decision models (Q8_0, same server) are included for context.
Seeds: text 1914503111, vision 363099055, Typed-Decisions 1530266809. 30 rows per vision benchmark and 150 Typed-Decisions states are small samples: differences of 1 to 2 points there are noise.
| Model | Text accuracy | Vision index | Typed-Decisions |
|---|---|---|---|
| Reference (Transformers, BF16) | 0.740 | 61.9 | 0.704 |
| Q8_0 | 0.740 | 61.5 | 0.706 |
| Q6_K | 0.740 | 61.9 | 0.708 |
| MACaQ-IQ4_XS | 0.736 | 62.4 | 0.719 |
| IQ4_XS (plain, not in this repo) | 0.735 | 63.1 | 0.727 |
| MACaQ-IQ2_XS | 0.672 | 57.7 | 0.693 |
| Liquid d1-2B (Q8_0) | 0.673 | 59.8 | 0.616 |
| Liquid d1-3B (Q8_0) | 0.655 | 46.3 | 0.672 |
| Liquid d1-Omni-2B (Q8_0) | 0.674 | 51.3 | 0.622 |
Same pick: the file's top answer equals the reference's. Gaps: largest difference of an option's probability, per question.
| File | Text same pick (4,484 q.) | Text p90 gap | Text largest gap | Vision same pick | Vision p90 gap | Vision largest gap |
|---|---|---|---|---|---|---|
| Q8_0 | 99.7% | 0.006 | 0.165 | 295/300 | 0.023 | 0.077 |
| Q6_K | 99.6% | 0.010 | 0.246 | 294/300 | 0.029 | 0.161 |
| MACaQ-IQ4_XS | 98.8% | 0.025 | 0.383 | 287/300 | 0.065 | 0.332 |
| IQ4_XS (plain, not in this repo) | 98.7% | 0.038 | 0.527 | 286/300 | 0.091 | 0.300 |
| MACaQ-IQ2_XS | 95.0% | 0.164 | 0.926 | 248/300 | 0.297 | 0.834 |
| Benchmark | Questions | Reference | Q8_0 | Q6_K | MACaQ-IQ4_XS | IQ4_XS (plain, not in this repo) | MACaQ-IQ2_XS | Liquid d1-2B (Q8_0) | Liquid d1-3B (Q8_0) | Liquid d1-Omni-2B (Q8_0) |
|---|---|---|---|---|---|---|---|---|---|---|
| ACOS-category-sentiment | 2412 | 0.98 | 0.98 | 0.98 | 0.98 | 0.99 | 0.98 | 0.99 | 0.99 | 0.99 |
| ANLI | 13 | 0.54 | 0.54 | 0.46 | 0.46 | 0.54 | 0.62 | 0.31 | 0.62 | 0.46 |
| API-Bank-tool-selection | 13 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.77 | 0.92 | 0.77 | 0.85 |
| ARC-Challenge | 12 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.83 | 0.83 | 0.75 |
| ARC-Easy | 12 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.83 | 1.00 | 0.83 |
| Amazon-ESCI | 12 | 0.83 | 0.83 | 0.75 | 0.83 | 0.75 | 0.75 | 0.83 | 0.67 | 0.83 |
| BANKING77 | 13 | 0.77 | 0.77 | 0.77 | 0.77 | 0.77 | 0.69 | 0.77 | 0.85 | 0.77 |
| BBH fixed-option tasks | 12 | 0.83 | 0.83 | 0.83 | 0.83 | 0.83 | 0.58 | 0.67 | 0.75 | 0.67 |
| BFCL-tool-selection | 38 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.97 | 0.95 | 0.95 |
| BPoMP-original-limerick | 40 | 0.85 | 0.85 | 0.85 | 0.82 | 0.88 | 0.70 | 0.88 | 0.75 | 0.82 |
| BRIGHT-retrieval | 20 | 0.65 | 0.65 | 0.65 | 0.65 | 0.70 | 0.55 | 0.60 | 0.65 | 0.80 |
| CFColor-preference | 13 | 0.69 | 0.77 | 0.77 | 0.62 | 0.85 | 0.69 | 0.77 | 0.62 | 0.77 |
| CLINC150+OOS | 13 | 0.92 | 0.92 | 0.92 | 0.92 | 0.85 | 0.77 | 0.92 | 0.77 | 0.85 |
| CLadder | 12 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.83 | 0.83 | 0.58 | 0.75 |
| CRUXEval-output-choice | 12 | 0.83 | 0.75 | 0.83 | 0.75 | 0.75 | 0.58 | 0.42 | 0.50 | 0.50 |
| ChessBench-legal-move | 12 | 0.08 | 0.08 | 0.08 | 0.08 | 0.08 | 0.00 | 0.25 | 0.17 | 0.25 |
| ContractNLI | 221 | 0.88 | 0.88 | 0.88 | 0.86 | 0.88 | 0.84 | 0.83 | 0.85 | 0.85 |
| FinEntity-entity-given | 26 | 0.85 | 0.85 | 0.85 | 0.85 | 0.88 | 0.81 | 0.92 | 0.92 | 0.92 |
| GPQA-Diamond | 13 | 0.69 | 0.69 | 0.62 | 0.77 | 0.69 | 0.38 | 0.38 | 0.46 | 0.46 |
| GSM8K-10 | 12 | 0.58 | 0.58 | 0.58 | 0.58 | 0.58 | 0.50 | 0.08 | 0.08 | 0.25 |
| GSM8K-4 | 12 | 0.83 | 0.83 | 0.83 | 0.83 | 0.83 | 0.75 | 0.50 | 0.33 | 0.33 |
| Habermas-consensus | 12 | 0.50 | 0.50 | 0.50 | 0.50 | 0.42 | 0.50 | 0.50 | 0.58 | 0.50 |
| HellaSwag | 12 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.83 | 0.83 | 0.92 | 0.83 |
| HoVer claim verification | 12 | 0.92 | 0.92 | 0.92 | 0.92 | 0.92 | 0.75 | 0.92 | 0.83 | 0.92 |
| Home-Appliance | 234 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.99 | 0.98 | 0.98 | 0.98 |
| Humicroedit | 13 | 0.62 | 0.62 | 0.69 | 0.62 | 0.69 | 0.62 | 0.54 | 0.62 | 0.69 |
| MMLU | 13 | 0.85 | 0.85 | 0.92 | 0.85 | 0.85 | 0.62 | 0.77 | 0.69 | 0.77 |
| MMLU-Pro | 12 | 0.67 | 0.67 | 0.67 | 0.75 | 0.67 | 0.75 | 0.42 | 0.58 | 0.58 |
| MuSR | 12 | 0.75 | 0.75 | 0.75 | 0.75 | 0.75 | 0.50 | 0.50 | 0.58 | 0.67 |
| NLI4CT-2024 | 12 | 0.92 | 0.92 | 0.92 | 0.92 | 0.83 | 0.67 | 0.83 | 0.75 | 0.92 |
| New Yorker caption matching | 12 | 0.83 | 0.83 | 0.83 | 0.83 | 0.75 | 0.75 | 0.75 | 0.42 | 0.67 |
| POP909-chord-pitch-class | 13 | 0.23 | 0.23 | 0.23 | 0.23 | 0.15 | 0.00 | 0.62 | 0.38 | 0.62 |
| PhishNChips phishing decisions | 48 | 0.83 | 0.83 | 0.83 | 0.83 | 0.85 | 0.83 | 0.79 | 0.62 | 0.58 |
| RAGTruth response-level hallucination | 12 | 0.92 | 0.92 | 0.92 | 1.00 | 0.92 | 0.83 | 0.92 | 0.92 | 0.83 |
| RouterBench-0shot | 26 | 0.04 | 0.04 | 0.04 | 0.04 | 0.08 | 0.04 | 0.04 | 0.04 | 0.00 |
| RouterBench-5shot | 26 | 0.12 | 0.12 | 0.12 | 0.08 | 0.12 | 0.04 | 0.04 | 0.00 | 0.00 |
| SATA-Bench | 136 | 0.90 | 0.90 | 0.90 | 0.90 | 0.93 | 0.83 | 0.85 | 0.85 | 0.82 |
| SGD-service-given-intent | 13 | 0.85 | 0.85 | 0.85 | 0.85 | 0.85 | 0.92 | 0.85 | 0.77 | 0.62 |
| SimpleBench | 10 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.20 | 0.00 | 0.10 | 0.00 |
| ToolRet-retrieval | 20 | 0.95 | 0.95 | 0.95 | 0.95 | 0.95 | 0.95 | 1.00 | 0.95 | 0.90 |
| VAST | 12 | 0.67 | 0.67 | 0.67 | 0.67 | 0.67 | 0.50 | 0.42 | 0.67 | 0.33 |
| When2Call MCQ | 12 | 0.83 | 0.83 | 0.83 | 0.83 | 0.83 | 0.75 | 0.83 | 0.83 | 0.92 |
| WinoGrande | 12 | 0.92 | 0.92 | 0.92 | 0.83 | 0.83 | 0.67 | 0.67 | 0.83 | 0.50 |
| iSarcasmEval-A-Ar | 4 | 0.75 | 0.75 | 0.75 | 0.75 | 0.75 | 0.75 | 0.75 | 0.25 | 0.75 |
| iSarcasmEval-A-En | 3 | 0.67 | 0.67 | 0.67 | 0.67 | 0.67 | 1.00 | 0.67 | 1.00 | 1.00 |
| iSarcasmEval-B-En | 30 | 0.90 | 0.90 | 0.90 | 0.90 | 0.90 | 0.97 | 0.97 | 0.87 | 0.93 |
RouterBench (cost-aware quality in the kit) and SimpleBench (10 trick questions, not counted in the index) are near zero for every model with this metric.
Vision: accuracy per benchmark| Benchmark | Chance | Reference | Q8_0 | Q6_K | MACaQ-IQ4_XS | IQ4_XS (plain, not in this repo) | MACaQ-IQ2_XS | Liquid d1-2B (Q8_0) | Liquid d1-3B (Q8_0) | Liquid d1-Omni-2B (Q8_0) |
|---|---|---|---|---|---|---|---|---|---|---|
| CV-Bench | 0.35 | 0.77 | 0.73 | 0.73 | 0.73 | 0.73 | 0.70 | 0.87 | 0.80 | 0.77 |
| BLINK | 0.35 | 0.70 | 0.70 | 0.70 | 0.70 | 0.67 | 0.63 | 0.57 | 0.50 | 0.57 |
| RealWorldQA | 0.39 | 0.67 | 0.70 | 0.70 | 0.70 | 0.73 | 0.77 | 0.77 | 0.63 | 0.73 |
| CharXiv | 0.25 | 0.83 | 0.80 | 0.83 | 0.80 | 0.83 | 0.77 | 0.70 | 0.63 | 0.63 |
| InfographicVQA | 0.25 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 0.93 | 0.90 | 0.77 | 0.87 |
| Mind2Web | 0.25 | 0.87 | 0.87 | 0.87 | 0.87 | 0.87 | 0.83 | 0.73 | 0.60 | 0.73 |
| KIE (CORD+FUNSD) | 0.43 | 0.93 | 0.93 | 0.93 | 0.93 | 0.93 | 0.90 | 0.90 | 0.93 | 0.83 |
| Moderation (Hateful Memes) | 0.50 | 0.57 | 0.57 | 0.57 | 0.60 | 0.60 | 0.47 | 0.67 | 0.53 | 0.53 |
| R-Bench-M | 0.17 | 0.27 | 0.27 | 0.27 | 0.27 | 0.30 | 0.37 | 0.40 | 0.30 | 0.40 |
| MMMU-Pro vision | 0.13 | 0.50 | 0.53 | 0.50 | 0.57 | 0.53 | 0.60 | 0.40 | 0.23 | 0.33 |
| Workflow | Questions | Reference | Q8_0 | Q6_K | MACaQ-IQ4_XS | IQ4_XS (plain, not in this repo) | MACaQ-IQ2_XS | Liquid d1-2B (Q8_0) | Liquid d1-3B (Q8_0) | Liquid d1-Omni-2B (Q8_0) |
|---|---|---|---|---|---|---|---|---|---|---|
| agent_trace_observability | 160 | 0.581 | 0.588 | 0.600 | 0.606 | 0.625 | 0.606 | 0.544 | 0.525 | 0.550 |
| customer_service | 210 | 0.748 | 0.752 | 0.743 | 0.738 | 0.762 | 0.729 | 0.633 | 0.700 | 0.657 |
| invoice_processing | 170 | 0.753 | 0.741 | 0.753 | 0.776 | 0.771 | 0.724 | 0.647 | 0.776 | 0.653 |
| security_incidents | 210 | 0.733 | 0.743 | 0.738 | 0.757 | 0.752 | 0.714 | 0.638 | 0.686 | 0.629 |
AUC: the chance that an altered image gets a higher "yes" than an unaltered one (1 = perfect, 0.5 = chance).
| Task | Reference | Q8_0 | Q6_K | MACaQ-IQ4_XS | IQ4_XS (plain, not in this repo) | MACaQ-IQ2_XS | Liquid d1-2B (Q8_0) | Liquid d1-3B (Q8_0) |
|---|---|---|---|---|---|---|---|---|
| pixelated (8, 16 px) | 0.901 | 0.903 | 0.901 | 0.914 | 0.897 | 0.919 | 0.857 | 0.882 |
| pixelated (4 px) | 0.888 | 0.889 | 0.887 | 0.890 | 0.880 | 0.851 | 0.947 | 0.949 |
| blurry (sigma 3) | 0.997 | 0.997 | 0.997 | 0.997 | 0.997 | 0.996 | 0.986 | 0.989 |
| blurry (sigma 1.5) | 0.944 | 0.941 | 0.942 | 0.936 | 0.941 | 0.935 | 0.892 | 0.924 |
| blurry, concept art | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| grayscale | 0.994 | 0.995 | 0.994 | 0.993 | 0.990 | 0.987 | 0.983 | 0.977 |
| grayscale, concept art | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
| perspective floor | 0.950 | 0.949 | 0.953 | 0.950 | 0.937 | 0.891 | 0.862 | 0.922 |
| upside down, concept art | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 | 0.999 | 0.985 |
| repeats: texture < its 3x3 tiling | 91% | 92% | 91% | 93% | 91% | 88% | 48% | 73% |
| material: same pick when mirrored | 96% | 95% | 97% | 96% | 97% | 96% | 95% | 96% |
Material pick equal to the reference's on the unaltered textures: Q8_0 99%, Q6_K 98%, MACaQ-IQ4_XS 98%, IQ4_XS (plain, not in this repo) 98%, MACaQ-IQ2_XS 95%.
RTX 4070 Ti SUPER (16 GB), llama.cpp b12359 + the patch, CUDA. 290 images (264 game textures at 256 px, 26 concept art pictures at 768 px), 4 questions each (two choices of 6 options, two of 2), 8 requests in flight, -np 8 -c 32768 -b 4096 -ub 4096; images encoded before timing, 24-image warm-up, two timed passes. Latency: 40 images one at a time. VRAM: weights, KV cache and compute buffers. The GPU had light background load from other programs, so differences under about 5% are noise.
| Model | Images/s (pass 1, 2) | Questions/s | Latency median / p90 | VRAM |
|---|---|---|---|---|
| Reference (Transformers, BF16) | 3.9, 3.9 | 16 | 276 / 528 ms | 9.9 GiB |
| Q8_0 | 4.8, 4.8 | 19 | 226 / 492 ms | 7.3 GiB |
| Q6_K | 4.3, 4.3 | 17 | 251 / 534 ms | 6.5 GiB |
| MACaQ-IQ4_XS | 4.5, 4.6 | 18 | 238 / 517 ms | 5.5 GiB |
| IQ4_XS (plain, not in this repo) | 4.9, 5.0 | 20 | 226 / 473 ms | 5.5 GiB |
| MACaQ-IQ2_XS | 4.0, 4.0 | 16 | 278 / 575 ms | 4.6 GiB |
| Liquid d1-2B (Q8_0) | 12.1, 12.1 | 48 | 109 / 227 ms | 3.7 GiB |
| Liquid d1-3B (Q8_0) | 9.4, 9.5 | 38 | 131 / 211 ms | 4.6 GiB |
The format barely changes the speed (4.0 to 5.0 images/s): the time is not in the weight matmuls. The server already batches the prompts of several slots, but the GPU is busy only about 60% of a run, every question is its own prompt (non-causal attention, no prompt sharing) and the image is encoded again for each question, one slot after the other. The context size makes no difference (5.2 to 5.3 images/s from 4k to 131k). The MACaQ rotation costs about 8% (MACaQ-IQ4_XS against a plain llama.cpp IQ4_XS).
PplxDeciderV4Model): the readout rows stored as an LM head on the answer-code
tokens, the prompt template of protocol v4, non-causal attention. BF16 and Q8_0 keep llama.cpp's fused attn_qkv_gate /
ssm_beta_alpha layout; Q6_K and the MACaQ files come from an unfused BF16 (MACaQ transforms attn_qkv, attn_gate,
ssm_beta and ssm_alpha one by one); the patched llama.cpp loads both layouts.macaq.py quantize --format IQ4_XS|IQ2_XS --had-block 64 --context 512 --ubatch 512 --chunks 256 (MACaQ 3810efc),
with the patched llama-imatrix / llama-quantize. 248 linears transformed and refined (median Hessian-weighted error 22% lower for
IQ4_XS, 27% for IQ2_XS).protocol.py:
60% Decision Index text questions (stratified by family), 20% Typed-Decisions train split, 20% Decision Index vision questions
without their images. None of the benchmark rows above (or their groups) is in it. It is not distributed: some sources do not allow it.llama-quantize --pure IQ4_XS with an imatrix from 32k tokens of
game-texture classification prompts, the same embedding (q4_K) and head (q6_K) types as the MACaQ file: a same-size comparison
for MACaQ. It is less faithful to the reference than MACaQ-IQ4_XS, so only the MACaQ version is distributed.Apache-2.0, as the base model: LICENSE and NOTICE are Perplexity's. These files are modified (converted, quantized) versions of
pplx-decider-flash-v1.3. The llama.cpp branch and patch are changes to llama.cpp (MIT). MACaQ is Apache-2.0. Benchmark data from the Decision Index kit (MIT), its
public sources (each under its own terms) and Typed-Decisions (Apache-2.0); no benchmark data is redistributed here.
perplexity-ai/pplx-decider-flash-v1.3