basal-1.5-mini-MLX-fp4

4-bit (nvfp4) MLX version of basal-1.5-mini (1.5B) for Apple Silicon. Same model, same prompt format and the same typed answers (choice, noul, score) with a probability for every option; see the main card for the description, benchmarks, limitations and license.

⚠ Quality warning. The 4-bit MLX port of basal-1.5-mini loses about 3.6 accuracy points against bf16 (0.882 vs 0.918) and its confidence thresholds are not certified. For mini, use -MLX-8bit or GGUF (Q8_0 / Q4_K_M), whose accuracy stays within noise of bf16.

Agreement with the bf16 model: 0.908; accuracy change −0.036 (0.882 vs 0.918) (500 held-out Polish development decisions, both option orders, calibrated, Apple M4 Pro; agreement = same top option as the reference (the same weights in PyTorch bf16; for basal-1.5-max, GGUF Q8_0 with llama.cpp); latency = median per decision (one question, both option orders) through the basal server path). Latency: 197 ms per decision on Apple M4 Pro, 24 GB. Memory: 1.0 GB. Accuracy here is on a 500-decision sample, to check that the port matches its reference; to compare model sizes, see the benchmark table of the main card.

Weights: MLX nvfp4 decoder layers (4-bit floating point E2M1, group 16, FP8 scales); the token embeddings and the output head stay in bf16, because the decision is read from the output head's logits of the option letters.

Files: the MLX weights and config, tokenizer and chat template, CALIBRATION.json and CALIBRATION.mlx.json (this port's temperatures, used automatically by --mode mlx), basal.json.

Quick start

uv venv --python 3.12 ~/basal-mlx && source ~/basal-mlx/bin/activate
uv pip install "basal[mlx] @ https://github.com/rkinas/basal/archive/refs/tags/v1.5.0.tar.gz"
basal-serve --model Remek/basal-1.5-mini-MLX-fp4 --mode mlx --port 8000

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.",
  "questions": {"dept": {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",
    "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}}}}'

The basal server renders the model's own prompt, asks both option orders, applies the calibration and supports multi, act and "facts": "auto"; the state of a request is computed once into a KV cache shared by all its questions. Evidence spans are not available with MLX (use the bf16 model with the CUDA engine).

Engines

The status column refers to basal-1.5 (4.5B); basal-1.5-mini uses the same engines and API, and its own measurements are listed where available.

basal-serve is the decision layer in every row. vLLM, SGLang, MLX, Ollama and llama.cpp run the weights; the typed answers (a calibrated probability for each of your options, both option orders, the state shared by several questions, multi, act and facts) come from basal-serve --mode <engine> in front of them, with the same HTTP API in every mode. A plain ollama run or llama-cli gives free text only. Evidence spans need the PyTorch engine (fast, eager); vLLM and SGLang start with a 4,096-token context (longer states are refused); raise it with --max-len (e.g. 32768 for long documents). Install and first request for each engine: quick start.

Calibration

CALIBRATION.json carries the temperatures and confidence thresholds of the bf16 model. This port's probabilities are flatter than the bf16 model's, so it ships CALIBRATION.mlx.json (used automatically by --mode mlx): the bf16 temperatures × 0.94 for choice and × 0.84 for yes/no questions, fitted on separate development decisions (expected calibration error 0.059 → 0.046). Its 1% and 5% confidence thresholds are copied from the bf16 model and are not certified for this port: the temperatures were refitted, so the bf16 model's coverage and error figures do not carry over. Refit the thresholds on your own labelled requests before automating decisions with them.

License

Apache-2.0, like basal-1.5-mini.