Eikos-4B-FP8
FP8 build of Eikos-4B: weights in FP8 (per channel) with dynamic FP8 activations (per token), made with llm-compressor (FP8_DYNAMIC, compressed-tensors format). No calibration data.
The vision tower, MTP weights, embeddings and LM head are kept in higher precision. The prompt format, the
letter readout and the calibration (calib.json, T = 1) are the same as the bf16 model.
Eikos answers typed decisions (yes/no, one of N options, ordinal scores) about a given state in one forward pass, with a calibrated probability for every option. See the Eikos-4B card for what the model does, how it was trained, the full evaluation and its limitations.
Use
Requires vLLM ≥ 0.30.0. Older builds return wrong answers when several long requests are batched together on this hybrid (Gated DeltaNet) architecture.
hf download caiovicentino1/Eikos-4B-FP8 --local-dir Eikos-4B-FP8
bash Eikos-4B-FP8/serve_vllm.sh $PWD/Eikos-4B-FP8 8001 # vLLM engine: letter readout + hybrid prefix cache
python Eikos-4B-FP8/serve.py --model $PWD/Eikos-4B-FP8 --vllm-url http://127.0.0.1:8001 --port 8000 # HTTP API on :8000
The HTTP API, agent sessions and the question types are the same as for Eikos-4B.
Images (serve v1.3): this build reads images too; its vision tower is kept in bf16. Send them in
"images", as multipart files or inside the state: see Images for the
format, the limits and the full results. Zero-shot, on the same public items for both builds:
| 300 items per set | Eikos-4B (bf16) | Eikos-4B-FP8 |
|---|---|---|
| MME (yes/no) | 89.7% | 89.3% |
| MMStar | 67.0% | 67.3% |
| SEED-Bench-2-Plus | 73.7% | 74.0% |
| ScreenSpot-v2, no marks | 55.7% | 56.3% |
| ScreenSpot-v2, 4×4 grid drawn | 53.3% | 54.3% |
| ScreenSpot-Pro, no marks | 52.7% | 51.7% |
| Same answer as bf16: all / confident (≥0.9) | — | 95.2% / 100.0% |
Images are harder than our text suites: the bf16 model is at least 90% confident on only 21% of these items. Of the 87 answers (out of 1,800) that differ from bf16, 99% are on items where bf16 itself was below 0.7 confidence.
Validation against bf16
Same 7,371 items for both builds (7 suites, never used in training), vLLM 0.30 with batching and prefix cache on. The release gate was fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.
| Eikos-4B (bf16) | Eikos-4B-FP8 | |
|---|---|---|
| Size | 9.3 GB | 5.8 GB |
| JevBench public — original / hard | 91.7 / 73.9 | 90.3 / 73.0 |
| DecisionBench — medium / hard | 77.1 / 66.6 | 78.5 / 66.6 |
| General battery (9 tasks) | 75.7 | 75.8 |
| Finance (CUAD, sentiment, FinQA-judge) | 74.7 | 75.2 |
| Trade rules — seen / unseen | 74.5 / 76.0 | 74.7 / 76.0 |
| Compositional rules — same type / new domain / rulebooks | 96.0 / 91.5 / 91.7 | 95.6 / 91.8 / 91.3 |
| ECE (lower is better) | 0.033 | 0.031 |
| ≥0.90 confidence: decides / error | 34.7% / 2.5% | 34.8% / 2.5% |
| Same answer as bf16 (all / confident ≥0.9) | — | 97.3% / 100.0% |
It passes our release gate.
License
MIT for our contributions (LICENSE). The base model, Qwen3.5-4B, is Apache-2.0 (LICENSE-Qwen);
attributions are in NOTICE. Not legal, tax or investment advice.