basal-omni-1.0-max-MLX (experimental)

An MLX 8-bit port of basal-omni-1.0-max (12B, base Gemma 4 12B) for Apple Silicon. Same model and the same typed-decision API through basal-serve --mode mlx. Text only: the image and audio encoders are not in this folder; for images and audio use the bf16 model.

The port passes every bar of the basal port gate against bf16 (agreement, agreement where bf16 is sure, accuracy, ECE), and ships its own calibration, fitted on this port's readouts.

Measured against bf16

Agreement is the share of decisions where the port picks the same option as the bf16 release model, served as released (--mode eager, bf16, one RTX PRO 6000, CUDA), on every 8th decision of the development set dev v2 (1,325 of 10,606), measured on an Apple M4 Pro (24 GB) with the engine's --mode mlx (mlx 0.32.4, mlx-lm 0.32.2), 8 requests in flight. Bars (the basal port gate for this tier): agreement on 2-10 options >= 0.9552, agreement where bf16's margin >= 0.2 >= 0.9893, accuracy within 1 point, ECE at most +0.01.

• 1,039 decisions with 2-10 options (950 of them with bf16's margin >= 0.2); over all 1,261 measured decisions (up to 255 options) the agreement is 0.9841. 64 of the 1,325 requests failed with a Metal out-of-memory error on the 24 GB Mac (8 in flight, in the stretch of long prompts) and are left out.
• Tokenization: the MLX folder renders and tokenizes the basal prompts exactly as the bf16 release (2,000 of 2,000 development prompts; the folder carries the release's own tokenizer and chat template).
• Automation with this folder's CALIBRATION.mlx.json (fitted on this port's readouts with --mode mlx; calibration test: every 2nd decision, 4,280 decisions, accuracy 0.909): at 1% target error 35.6% of decisions are decided automatically (observed error 0.5%); at 5%, 85.2% (3.6%).
• Latency (M4 Pro, basal-loadtest, one request at a time, median): 453 ms for one question, 1504 ms for five questions about one state.
• The port is a decision-layer conversion of the same model, not a new model; its benchmark results (Werdykt v1 0.823, the Polish and English sets) are on basal-omni-1.0-max. This folder is not scored separately on them.

Use

Install the basal engine 2.2.0 or newer with the mlx extra (Apple Silicon, Python 3.11 or newer) and serve the folder with --mode mlx:

uv venv --python 3.12 --allow-existing ~/basal-mlx && source ~/basal-mlx/bin/activate
uv pip install "basal[mlx] @ https://github.com/rkinas/basal/archive/refs/tags/v2.2.0.tar.gz"
basal-serve --model Remek/basal-omni-1.0-max-MLX --mode mlx --port 8000

Then ask questions exactly as with basal-omni-1.0-max (choice, noul, score, multi, act, tool, and the task types; up to 255 options).

• Calibration: --mode mlx loads CALIBRATION.mlx.json, fitted on this port. CALIBRATION.json is the bf16 model's calibration. You can refit either on your own labelled data with basal-calibrate.
• Memory: the weights take 13.6 GB of unified memory; long states need more on top. On the 24 GB M4 Pro, 64 of 1,325 requests ran out of GPU memory with 8 in flight among long prompts, while the calibration readouts, one batch at a time, ran 9,187 of 9,187 items; for long documents with many requests in flight, use a Mac with more memory.
• One option order. The shipped calibration is fitted for one option order (the default).

How it was made

• mlx-lm 0.32.2's converter (load, quantize, save) on the bf16 release model: MLX affine 8-bit, group size 64, for the linear layers of the decoder blocks; the token embeddings (which are also the output head the decision is read from, in fp32) and the norms stay bf16. mlx-lm keeps the text decoder only (the image and audio encoders are left out). The rotary and global-attention settings are written into config.json in the keys mlx-lm reads.
• CALIBRATION.mlx.json: the bf16 model's method and splits on this port's readouts (--mode mlx, one option order, every 2nd item of each split): per-type and per-bucket temperatures and certified thresholds for 1% and 5% error; a looser target never gets a stricter threshold.

Files

• model*.safetensors, config.json: the MLX weights and their configuration.
• Tokenizer, chat_template.jinja: as in the bf16 model (the basal server renders the prompt with them).
• CALIBRATION.mlx.json, CALIBRATION.json: the calibration fitted on this port, and the bf16 model's.
• basal.json: prompt format 2 and the readout protocol.

Limitations

• Experimental, as the bf16 model: validate it on your own documents before relying on it.
• Text only (see above).
• Quantisation changes some decisions where the model is unsure (see the table); decisions it is sure about are almost unchanged.
• Exact arithmetic and thresholds are a weak spot of one-pass decisions: compute them in code.
• Decisions with serious consequences for people should be reviewed by a person.

Citation

@software{kinas2026basal20,
  title   = {basal-2.0: typed-decision models and inference engine for Polish and English},
  author  = {Kinas, Remigiusz},
  year    = {2026},
  version = {2.0.0},
  url     = {https://github.com/rkinas/basal}
}

License and attribution

Apache-2.0. Converted from basal-omni-1.0-max, fine-tuned from google/gemma-4-12B-it (Apache-2.0).