mlx-community/clef-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune and serve models on Apple Silicon. Decision models guide · All OptiQ quants

A 4-bit mixed-precision MLX quant of Cloudflare/clef, Cloudflare's 27B decision model. A decision model reads a piece of state and a set of typed questions (yes/no, choice, score) and returns a probability for every option of every question in one forward pass. There is no generated text and nothing to parse. The request and response match the Jev / SystemOne API.

The backbone is quantized with per-layer bit-widths: sensitive layers at 8-bit, the rest at 4-bit. The joint schema head and the vision tower are kept at full precision. Size on disk is about 20.6 GB.

Typed Decisions result

Scored on the full 400-case Typed Decisions test split (2,000 decisions) with the benchmark's own metrics, text only, MLX on an Apple M3 Max. The comparison is the stock uniform 4-bit quant of the same model, mlx-community/clef-4bit, scored the same way.

The probabilities sit a little closer to the reference answers (lower KL and Brier). Accuracy is the same within noise. The gain over uniform 4-bit is small; the point of this quant is that it loses nothing and runs through OptiQ's decision endpoint.

Use

Serve it with OptiQ and the decision endpoint turns on by itself:

pip install mlx-optiq
optiq serve --model mlx-community/clef-OptiQ-4bit

curl http://127.0.0.1:8080/v1/systemone -d '{
  "model": "clef",
  "state": "Checkout has been failing for every customer for the last hour.",
  "questions": {
    "urgent": {"type": "noul", "instructions": "Is this urgent?"},
    "team": {"type": "choice", "criteria": {"billing": "Payments", "technical": "Outages and errors"}}
  }
}'

Or from Python:

from optiq.decision import DecisionModel

model = DecisionModel.load("mlx-community/clef-OptiQ-4bit")
print(model.predict({
    "state": "Checkout has been failing for every customer for the last hour.",
    "questions": {"urgent": {"type": "noul", "instructions": "Is this urgent?"}},
}))

Text input only for now: a request that carries images or video is refused with a clear error rather than answered without them. The decision models guide covers the three question types, errors, context length and training your own with optiq lora train.

Quantization details

The bit allocation reuses the measured recipe of the Qwen3.8-27B OptiQ quant, the backbone this model was post-trained from.

See also clef-flash-OptiQ-4bit (Clef-Flash, 9B).

License

Apache-2.0, as the base model. Clef is by Cloudflare; see the base model card for how it was trained and evaluated.