basal-2.0-4.5B

basal-2.0 (4.5B) is part of basal-2.0, a family of typed-decision models for Polish and English; it is the main model: the best balance of accuracy and speed. Built on Bielik-4.5B-v3.0-Instruct. Trained further from earlier basal checkpoints.

What it is. The model reads a state (a message, a document, retrieved passages, an agent trace, a web page as JSON) and answers a typed question about it (choice, yes/no noul, score, multi, act, tool) by returning a calibrated probability for each allowed answer, in one forward pass, without generating text. The answer never falls outside the options you give, and the probability says how sure the model is. Because the classes are described in the request, one model serves many tasks without retraining: routing, rule and policy checks, questions about long documents, RAG decisions, urgency and risk scores, judging answers, agent and tool decisions, and triage with a confidence threshold.

Each model also comes as -FP8 (NVIDIA Hopper and Blackwell), -GGUF (Q8_0 and Q5_K_M; Ollama, llama.cpp) and -MLX-8bit (Apple Silicon).

Intended use

• Good for: routing messages and documents to one of the options you describe; yes/no checks of rules and policies; urgency and risk scores; questions about long documents and retrieved passages (is the answer there, is it relevant); judging answers; choosing an agent's next action or tool (a tool question over OpenAI or MCP tool definitions picks the tool, not its arguments); triage that hands uncertain cases to a person (an act question with a defer action and max_error).
• Languages: Polish and English (the state and the question may be in either).
• Not for: writing text or chatting (the model only picks among the options you give); facts that are not in the state; decisions with serious consequences for people without human review.

Quick start

1. Install the basal engine v2.5.0 into a fresh uv environment (NVIDIA GPU; the cu128 torch needs a driver for CUDA 12.8 or newer; DGX Spark and B300 use the cu130 build, see docs/HARDWARE.md):

uv venv --python 3.12 --allow-existing ~/basal-env && source ~/basal-env/bin/activate
uv pip install torch==2.11.0 --index-url https://download.pytorch.org/whl/cu128
uv pip install "basal[fp8] @ https://github.com/rkinas/basal/archive/refs/tags/v2.5.3.tar.gz"

2. Start the server and wait for basal: model ... ready:

basal-serve --model Remek/basal-2.0-4.5B --mode fast --template-prefix off --port 8000

--template-prefix off is needed for this model (bf16 and -FP8): the prompt-template cache failed its check, and its numbers were measured without it.

3. Ask questions with curl. Several questions about one state go into one request and share the state:

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.",
  "questions": {
    "dept":   {"type": "choice", "instructions": "Do którego działu skierować zgłoszenie?",
               "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"}},
    "urgent": {"type": "noul", "instructions": "Czy klient nie może korzystać z usługi?"}}}'

Each answer has a probability for every option, the chosen key and the confidence: "dept": {"type": "choice", "choice": "<key>", "probabilities": {"cards": p, "online": p, "loans": p}, "confidence": p}.

To send uncertain decisions to a person, ask the same question as an act question with a defer action and "max_error": 0.05 (max_error works on act and tool questions only):

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła.",
  "questions": {
    "dept": {"type": "act", "instructions": "Do którego działu skierować zgłoszenie?",
             "criteria": {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"},
             "option_keys": "hide", "costs": {"wrong": 19, "defer": 1}, "max_error": 0.05}}}'

The answer is the action with the lowest expected cost (an option key, or defer). Below the confidence threshold certified for 5% error it returns defer, names the refused action and says "deferred": true. "option_keys": "hide" shows the model the option descriptions only, as the calibration was fitted (with the keys shown, answers say "calibration": "unvalidated"); keep the cost of a wrong answer moderate, because a large one defers by expected cost alone, before max_error is applied.

Or from Python (the package includes a small client):

from basal.client import Basal
b = Basal("http://127.0.0.1:8000")
state = "Klient: od wczoraj nie mogę zalogować się do bankowości internetowej, system pokazuje błąd hasła."
a = b.choice(state, "Do którego działu skierować zgłoszenie?",
             {"cards": "Reklamacje kart", "online": "Wsparcie bankowości elektronicznej", "loans": "Kredyty"})
print(a["choice"], a["confidence"])
print(b.yes_no(state, "Czy klient nie może korzystać z usługi?")["noul"])   # P(yes)

Many items from a file: basal-run --input items.jsonl --output answers.jsonl. API and features: README.

The same basal engine v2.5.0 also serves the experimental basal-omni-1.0 models (images and audio): README.

Results against basal-1.5

Measured once on the frozen release weights, bf16 (basal-serve --mode fast) on one H100, and published as measured, including where basal-2.0 is not better.

basal-1.5 is basal-1.5-4.5B, the previous release (Remek/basal-1.5-4.5B). Werdykt rows are the public board's rows (basal-1.5 served on SGLang, basal-2.0 by basal-serve v2.0.0 in its default mode), each release with its own calibration file, accuracy with --deterministic; automated at X% target error = a confidence threshold chosen on one half of the items for that error, applied to the other half. The speed pass is the board's (bf16, 16 clients, one H100 80GB; separate pods); the second speed row has basal-1.5 in basal-serve's default mode too. The rank is on the public Werdykt board (basal.si5.pl/benchmark), 94 systems, one row per system. All other rows: both models served by engine v2.0.0 (commit adc4c71) on the same items; differences are paired, with 95% bootstrap intervals. Polish general and the English corpus test were used to choose between training runs, so their numbers are development numbers. LEK questions: NASK-PIB Reassess, 2008-2026 sessions (CC BY-NC-SA 4.0), shown for the basal models only, with no comparison to other systems.

Accuracy by document length

Accuracy of basal-2.0-4.5B on Werdykt (5,000 items, Polish and English; a hidden set, so only these totals are shown) by the length of the state, in characters (a token is about 2 characters; 1,000 characters is about half a page). 72.9% overall. 95% Wilson intervals in brackets.

Short states are mostly rules and abstention items, which are harder for every basal model, so read the rows as accuracy per bucket, not as the effect of length alone. The long-document family keeps the task and varies only the length:

Run 2026-10-10 with basal-serve --mode fast on one NVIDIA H100 80GB HBM3 (bf16). The intervals are about the items, not about run-to-run noise.

By prompt length in tokens (held-out tests)

The held-out tests of the Results table by the length of the whole prompt as served (state, question and options; this model's tokenizer), with the Results table's predictor. Accuracy (items):

What is new since basal-1.5

• Prompt format 2. Up to 255 options per question (basal-1.5: 10), read in one forward pass with one option order by default (basal-1.5 read two orders).
• Training. Trained further from earlier basal checkpoints.
• Calibration. Temperatures per question type and option-count bucket, fitted on question families the model was not trained on.
• Engine v2.0.0. Compact batching (+19% decisions/s at 16 clients, +17% at 32, +8% at one client) (one H100, bf16, each change left out in turn); the prompt-template cache is off for this model (--template-prefix off: it failed this model's check). They need FlashAttention 3 (Hopper): the Werdykt rows on this card ran with FlashAttention 3; the release check, the port checks, the FP8 speed and the calibration ran with FlashAttention 2, without them. Engine v2.0.0 schedules long requests (over 2,048 tokens) behind short ones: in a mixed load at 32 clients short requests answer in 0.81 s instead of 1.38 s (p50), and long requests take about 7% longer (p50 1.96 s instead of 1.83 s); --scheduler off serves in arrival order.
• Thinking mode (experimental, off by default). See below.

Thinking mode (experimental)

Off by default, and we recommend leaving it off. With --think auto the model writes a short reasoning note before it decides, but only for questions whose calibrated confidence is below the type's certified 5% threshold; the decision is then read out again over the state and the note, with its own calibration file (CALIBRATION.think.json). The note is generated by a vLLM server that basal-serve starts in its own process on the think weights in think/. vLLM brings its own torch (vLLM 0.31: torch built for CUDA 13.0, which needs an NVIDIA driver 580 or newer), so install it in its own environment and point --think-python at that environment's Python; the server itself runs from the quick-start environment:

uv venv --python 3.12 --allow-existing ~/basal-vllm
uv pip install --python ~/basal-vllm/bin/python "basal[vllm] @ https://github.com/rkinas/basal/archive/refs/tags/v2.5.3.tar.gz"

source ~/basal-env/bin/activate
basal-serve --model Remek/basal-2.0-4.5B --mode fast --template-prefix off --think auto --think-python ~/basal-vllm/bin/python --port 8000
basal-serve --model Remek/basal-2.0-4.5B --mode fast --template-prefix off --think auto --think-tokens 128 \
    --think-python ~/basal-vllm/bin/python --port 8000      # shorter notes (default 256)
basal-serve --model Remek/basal-2.0-4.5B --mode fast --template-prefix off --port 8000   # off (the default: --think off)

Per request, on a --think auto server: "think": "auto" (the default), "off" (one-pass answer for this request) or "force" (every question of a think type thinks); "think_text": true returns the note. A question that thought answers with the second readout's probabilities and confidence plus "think": {"tokens", "ms", "calibration", "no_think": {...}}; every answer of a think server carries "meta": {"think": {"used", "tokens"}}. Multi questions, tool sets and requests whose state was truncated never think. On a server started without --think auto (the default) there is no think path.

Measured once on Werdykt, both rows on the same pod (one H100 80GB, default mode, 16 clients for the speed column):

On Werdykt, thinking lowered the macro score by 0.9 points (0.719 against 0.729), raised the calibration error (0.099 against 0.060) and cut throughput to 11.5 decisions/s from 31.2. Every other number on this card is the default one-pass mode. Try --think auto only with your own measurement on your own data.

Ports

Every port is checked against this bf16 model on the development set dev v2: agreement is the share of decisions where the port picks the same option. Bars for this tier: agreement 2-10 >= 0.9415, agreement at reference margin >= 0.2 >= 0.9863, accuracy within 1 point, ECE at most +0.01.

Calibration

For every question basal returns a probability for each answer; the confidence is the probability of the answer it picks. The probabilities are calibrated (a temperature per question type and option-count bucket) on question families that were never used for training, so they hold on new tasks but are cautious. Held-out numbers below are leave-one-family-out on the development and test splits.

On Werdykt (hidden, measured once, not used to choose the model) the calibration error is 0.060 (basal-1.5: 0.068). The calibration is a file, separate from the weights (CALIBRATION.json, one option order; CALIBRATION.o2.json, two orders), so you can swap it or refit it on your own data with basal-calibrate. Fitted with engine v2.0.0 (commit adc4c71).

Speed

One decision = one question, one option order. Hardware is named on every number.

• Werdykt speed pass (bf16, one H100 80GB): basal-2.0 32.2 decisions/s at 16 clients, c1 p50 25 ms (p95 188 ms) in basal-serve v2.0.0's default mode; basal-1.5 23.0 decisions/s at 16 clients, c1 p50 34 ms (p95 270 ms) on SGLang, as on the public board; in basal-serve's default mode basal-1.5 does 8.0 decisions/s at 16 clients, c1 p50 31 ms (p95 795 ms). Separate pods of the same GPU type.
• FP8 package: 38.6 dec/s c16, c1 p50 29 ms (H100); one NVIDIA H100 80GB (cluster node), one option order.
• Measured setup: the Werdykt rows (accuracy and the speed pass) ran on the Werdykt pods with FlashAttention 3 installed, which the engine needs for compact rows (an engine default; the template prefix stays off for this model); the release check, the port checks, the FP8 speed and the calibration fits ran with FlashAttention 2, so compact rows and the template prefix were off there. FP8 keeps FlashAttention 2 unless --attn fa3.
• Accuracy rows use --deterministic; default minus --deterministic accuracy on dev v2: bf16 +0.0000, FP8 -0.0025 (run 1; 32 requests in flight); --deterministic itself: bf16 200/200 identical at 16 clients, FP8 200/200 identical at 16 clients.

basal-1.5's other configuration. basal-1.5 also shipped FP8 checkpoints served by vLLM. On one RunPod H100 (median of 3 fresh servers; synthetic ~1.6k-token states and a 400-request mix, a different workload from the Werdykt speed pass) the two basal-1.5 configurations measured:

basal-2.0 was not measured on that machine, so the speed-ups in the results table hold only against the basal-1.5 rows of the Werdykt speed pass. For mixed traffic with long, repeated documents, measure both on your own load before you switch.

Evaluation notes

• Werdykt (hidden benchmark). Every training file after basal-1.5 was checked against the Werdykt guard list (exact state, rare 13-word shingles, source documents) and builds fail on any hit. This model continues earlier basal checkpoints back to basal-1.5, so it inherits basal-1.5's exposure to 7 boilerplate rows (effect at most 0.14 points on basal-1.5).
• PL-Real-300. Training rows of the basal checkpoints between basal-1.5 and basal-2.0 came from 27 of the PL-Real source judgments (their guard excluded PL-Real states, not the source documents); the 2.0 training files add 0. The primary PL-Real figure is therefore the doc-clean subset (158 items on the other judgments), for both models; the full 300 is shown next to it.
• Selection sets. Polish general and the English corpus test were used to choose between training runs, so their numbers are development numbers. PL-Holdout, PL-Real, LEK, JDI and Werdykt were not used to select basal-2.0.

Files

• model.safetensors, tokenizer, chat_template.jinja: the model
• CALIBRATION.json (one option order) and CALIBRATION.o2.json (two orders): per-type temperatures and confidence thresholds for 1% / 5% error
• think/ (the think weights) and CALIBRATION.think.json: the experimental thinking mode
• basal.json: prompt format 2 and readout protocol

How it works

The model receives a fixed chat prompt with the state, the question and coded options; the decision is the softmax over the next-token logits of the option codes only (one forward pass, fp32 code logits, no text generation), then the per-type calibrated temperature from CALIBRATION.json. Confidence thresholds were chosen on calibration data, before testing, for 1% or 5% error among accepted decisions.

Training

Fine-tuned from Bielik-4.5B-v3.0-Instruct for typed decisions in Polish and English, trained further from earlier basal checkpoints. LoRA stages, merged.

Limitations

• Validate on your own documents before relying on it: hidden and held-out sets do not cover every domain.
• Polish world knowledge of a small model is limited: put the relevant facts in the state. Legal rules change.
• Exact arithmetic and thresholds are a weak spot of one-pass decisions: compute them in code (or "facts": "auto").
• The calibration is cautious on unseen tasks: no 1% threshold is certified in the shipped files, so "max_error": 0.01 on an act or tool question defers every decision; use 0.05, or certify thresholds on your own data with basal-calibrate.
• act and multi over many labels are experimental. Up to 255 options per question (Ollama: 20); the context is 32,768 tokens. The 2.0 models ship no evidence head.
• Decisions with serious consequences for people should be reviewed by a person.

Citation

@software{kinas2026basal20,
  title   = {basal-2.0: typed-decision models and inference engine for Polish and English},
  author  = {Kinas, Remigiusz},
  year    = {2026},
  version = {2.0.0},
  url     = {https://github.com/rkinas/basal}
}

License and attribution

Apache-2.0. Fine-tuned from speakleash/Bielik-4.5B-v3.0-Instruct (Apache-2.0). Bielik is a project of SpeakLeash and ACK Cyfronet AGH.