basal-1.0-4.5B-FP8

FP8 version of basal-1.0-4.5B, produced with NVIDIA Model Optimizer post-training quantization (calibrated on 1,024 decision prompts from the calibration split; the output head stays in bf16 because the decision is read from its logits). For Hopper and Blackwell GPUs, served with vLLM through the basal engine (Quick start below).

📊 More results — accuracy and speed of basal-1.0 against Jev and other open decision models: jev-pl-benchmark (Polish score, English score, speed).

This checkpoint has not been benchmarked separately yet; expect results close to the on-the-fly FP8 mode (--mode fp8, same number format), whose speed and agreement with the fp32 reference are in the main model card and https://github.com/rkinas/basal/blob/main/docs/HARDWARE.md. CALIBRATION.json here carries the temperatures and confidence thresholds of the bf16 model, marked as not validated for this checkpoint (its decisions differ); refit them on your own labelled data before relying on the thresholds. See basal-1.0-4.5B for the description, benchmarks, limitations and license.

Quick start

1. Install into a fresh uv environment (no git needed):

uv venv --python 3.12 ~/basal-vllm && source ~/basal-vllm/bin/activate
uv pip install "basal[vllm] @ https://github.com/rkinas/basal/archive/refs/tags/v1.0.1.tar.gz"

A separate environment: vLLM installs its own torch. FP8 runs on Hopper and Blackwell GPUs.

2. Start the server and wait until it prints basal: model ... ready:

basal-serve --model Remek/basal-1.0-4.5B-FP8 --mode vllm --port 8000

vLLM loads the checkpoint and captures its graphs on the first start.

3. Ask a question (in a second terminal):

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej.",
  "questions": {"dept": {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",
    "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}}}}'

The response has a calibrated probability for every option, the chosen key and the confidence:

{"model": "basal-1.0-4.5B-FP8",
 "answers": {"dept": {"type": "choice", "choice": "online",
   "probabilities": {"cards": …, "online": …, "loans": …}, "confidence": …}},
 "usage": {"input_tokens": …, "output_tokens": 0, "questions": 1, "latency_ms": …}}

From Python (the basal package includes a client):

from basal.client import Basal
b = Basal("http://127.0.0.1:8000")
state = "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej."
a = b.choice(state, "Do którego działu skierować zgłoszenie?",
             {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"})
print(a["choice"], a["confidence"])                                      # chosen key and its probability
print(b.yes_no(state, "Czy klient zgłasza problem techniczny?")["noul"])  # P(yes)
print(b.score(state, "Jak pilne jest zgłoszenie?", ["niska", "średnia", "wysoka"])["score"])  # expected level 0-2

Many items at once: basal-run --input items.jsonl --output answers.jsonl (JSONL format). Several questions per request, option keys, early exit and the full response format: API.

Citation

@techreport{kinas2026basal,
  title       = {basal-1.0: Reliable, Highly Optimized Typed Decisions for Polish},
  author      = {Kinas, Remigiusz},
  institution = {ai5},
  year        = {2026},
  type        = {Technical report},
  doi         = {10.5281/zenodo.23022986},
  url         = {https://doi.org/10.5281/zenodo.23022986}
}