pplx-decider-flash-v1.2-ternary-mlx

Two D3 mixed-precision MLX builds of the text backbone of perplexity-ai/pplx-decider-flash-v1.2 (a multi-question decision/classification model). Text only.

These are the d3-ternary-* folders from the main repo Jakevin/pplx-decider-flash-v1.2-mlx (added in commits 64b14d6 and 92cfaeb), republished unchanged as a separate repo, together with the BF16 readout head and the runner/ code from the main repo, so this repo works on its own. The main repo also has the uniform 4-bit / 3-bit builds and the full evaluation details.

All numbers are copied from the main repo's model card (section "D3 mixed-precision variants"). They are agreement with the original unquantized BF16 model, run through the same non-causal runner, on a held-out set of 246 questions. For comparison, the main repo reports the uniform 4-bit g32 build (2.45 GiB) at 97.5 % and the uniform 4-bit g64 build (2.20 GiB) at 94.6 % on the same subset. The choice+noul subset has 204 questions, so one question is about 0.5 points, and differences of ±1–2 questions are within noise. The main repo notes that KL and confident flips are the more stable signals. Measured on an Apple M4 / 16 GB with a small sample, so validate on your own task.

About "ternary": it covers only about 1 % of parameters

These are not fully ternary models. D3 measures the KL of each layer group's output and then assigns bits per group (token embedding; per layer the token mixer, MLP gate+up and MLP down). Ternary was chosen for exactly one group: the layer-29 Gated-DeltaNet token mixer, which is about 1 % of the parameters (1.0 % in both builds). Everything else is 2/3/4/8-bit affine quantization. In the main repo's words, the bit mix (share of quantized parameters, including the embedding) is:

• 2.30g: ternary 1.0%, 2-bit 5.9%, 3-bit g64 20.0%, 3-bit g32 0.6%, 4-bit g64 39.7%, 4-bit g32 22.2%, 8-bit 10.6% (embedding 4-bit g64).
• 2.20g: ternary 1.0%, 2-bit 7.6%, 3-bit g64 18.4%, 3-bit g32 15.7%, 4-bit g64 28.9%, 4-bit g32 20.3%, 8-bit 8.1% (embedding 3-bit g32).

The ternary group is stored in MLX's standard 2-bit affine format (codes 0/1/2 → −s/0/+s), so stock MLX kernels run it with no custom kernel. Per-group formats are in each folder's d3_allocation.json and in config.json → quantization.

Files

readout.safetensors          # BF16 readout head [255, 2560], shared by both variants (identical to the main repo's file)
runner/                      # MLX runner code (copied unchanged from the main repo)
  decider_common.py          #   prompt/marker encoder + answer decoding
  mlx_decider.py             #   mlx-lm backbone loading, non-causal forward, readout
  run_mlx.py                 #   main repo's uniform-quantization eval script (needs the original BF16 checkpoint + eval_set.json; not needed for inference)
d3-ternary-2.30g/  d3-ternary-2.20g/
  model.safetensors, config.json, tokenizer.json, tokenizer_config.json, chat_template.jinja,
  decision_config.json, joint_config.json, d3_allocation.json

The variant folders do not contain their own readout.safetensors. They use the one at the repo root, the same layout as the main repo. This repo is self-contained, so you don't need to download anything from the main repo.

Usage (MLX / mlx-lm)

The weights load with mlx-lm (mlx_lm.utils.load_model, qwen3_5 layout). This is a classifier, though, not a chat model: there is no lm_head, so mlx_lm.generate / mlx_lm.chat are not meaningful. Use the bundled non-causal runner and readout head:

import sys
from huggingface_hub import snapshot_download

repo = snapshot_download("Jakevin/pplx-decider-flash-v1.2-ternary-mlx",
                         allow_patterns=["runner/*", "readout.safetensors", "d3-ternary-2.30g/*"])  # or "d3-ternary-2.20g/*"
sys.path.insert(0, f"{repo}/runner")
import mlx_decider as D
from decider_common import Encoder, answers

path = f"{repo}/d3-ternary-2.30g"
enc = Encoder(path)
model, _ = D.load_mlx_dir(path)        # mlx_lm.utils.load_model under the hood
W = D.load_readout(repo)               # BF16 readout from the repo root
req = {"state": "My integration keeps failing with a 401 since we rotated our API key.",
       "questions": {"team": {"type": "choice", "criteria": {"billing": None, "support": None, "sales": None}}}}
ids, markers, counts = enc.encode(req)
print(answers(req, D.readout_logits(model, W, ids, markers, counts), enc.temperature))

Requires mlx, mlx-lm (the main repo was tested with 0.32.4 / 0.32.2) and transformers (tokenizer only).

Provenance and license

This is a derivative work of perplexity-ai/pplx-decider-flash-v1.2, which is itself derived from Qwen/Qwen3.5-4B. It is distributed under the same Apache License 2.0, and the original LICENSE and NOTICE files are included unchanged. Modifications (vision tower dropped, weight renaming, quantization) are documented in the main repo's model card. This is not an official Perplexity release.