Eikos-27B-MLX-4bit
MLX 4-bit build of Eikos-27B: MLX weights quantized to 4 bits (group size 64; mlx_lm convert -q --q-bits 4 --q-group-size 64) for Apple Silicon.
The conversion keeps the text model only (no vision tower, no MTP weights), and the embeddings and LM head
are quantized like the other layers. The prompt format, the
letter readout and the calibration (calib.json, T = 1) are the same as the bf16 model.
Eikos answers typed decisions (yes/no, one of N options, ordinal scores) about a given state in one forward pass, with a calibrated probability for every option. See the Eikos-27B card for what the model does, how it was trained, the full evaluation and its limitations.
Use
pip install mlx mlx-lm
hf download caiovicentino1/Eikos-27B-MLX-4bit --local-dir Eikos-27B-MLX-4bit
python Eikos-27B-MLX-4bit/mlx_decide.py Eikos-27B-MLX-4bit # runs a few demo decisions
import sys; sys.path.insert(0, "Eikos-27B-MLX-4bit")
from decision_core import options_of
from mlx_decide import MLXDecider
d = MLXDecider("Eikos-27B-MLX-4bit")
q = {"type": "noul", "instructions": "Under rule 4.2, can this order be executed as submitted?",
"criteria": {"true": "complies with rule 4.2", "false": "breaches rule 4.2"}}
probs, n_tokens = d.dist("Order ticket #A-2231 ... Rule 4.2: ... Approvals on file: none.", q, options_of(q))
mlx_decide.py uses the same letter readout, prompt and calibration as the GPU builds. It also answers several
questions over one state in a single cached batch (dist_many_cached).
Text only: this build has no vision weights, so it does not read images. Images need a GPU build with vLLM and serve v1.3: see Images.
Memory: the weights take about 15 GB, so a Mac with 32 GB or more is recommended.
Validation against bf16
Same 7,371 items for both builds (7 suites, never used in training), MLX run with its CUDA backend on Linux. The release gate was fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.
| Eikos-27B (bf16) | Eikos-27B-MLX-4bit | |
|---|---|---|
| Size | 55.6 GB | 15.1 GB |
| JevBench public — original / hard | 100.0 / 82.0 | 100.0 / 83.8 |
| DecisionBench — medium / hard | 89.1 / 78.2 | 88.4 / 78.2 |
| General battery (9 tasks) | 82.6 | 81.8 |
| Finance (CUAD, sentiment, FinQA-judge) | 85.4 | 84.9 |
| Trade rules — seen / unseen | 85.4 / 87.4 | 86.1 / 85.7 |
| Compositional rules — same type / new domain / rulebooks | 95.6 / 94.3 / 95.3 | 95.8 / 94.2 / 95.7 |
| ECE (lower is better) | 0.043 | 0.041 |
| ≥0.90 confidence: decides / error | 44.1% / 2.5% | 42.9% / 1.9% |
| Same answer as bf16 (all / confident ≥0.9) | — | 96.6% / 99.9% |
It passes our release gate on accuracy and calibration, but not on answer agreement (96.6% of answers are the same as bf16; the gate asks for 97%). Most changed answers (92%) are on items where the bf16 model itself was unsure (confidence below 0.7); on decisions the bf16 model takes with confidence ≥0.9, 99.9% of answers are the same.
License
MIT for our contributions (LICENSE). The base model, Qwen3.8-27B, is Apache-2.0 (LICENSE-Qwen);
attributions are in NOTICE. Not legal, tax or investment advice.