Eikos-4B-INT4
INT4 (GPTQ) build of Eikos-4B: 4-bit weights (GPTQ W4A16, groups of 128) with bf16 activations, made with llm-compressor (compressed-tensors format), calibrated on 256 training items (never on evaluation data).
The vision tower, MTP weights, embeddings and LM head are kept in higher precision. The prompt format, the
letter readout and the calibration (calib.json, T = 1) are the same as the bf16 model.
Eikos answers typed decisions (yes/no, one of N options, ordinal scores) about a given state in one forward pass, with a calibrated probability for every option. See the Eikos-4B card for what the model does, how it was trained, the full evaluation and its limitations.
Use
Requires vLLM ≥ 0.30.0. Older builds return wrong answers when several long requests are batched together on this hybrid (Gated DeltaNet) architecture.
hf download caiovicentino1/Eikos-4B-INT4 --local-dir Eikos-4B-INT4
bash Eikos-4B-INT4/serve_vllm.sh $PWD/Eikos-4B-INT4 8001 # vLLM engine: letter readout + hybrid prefix cache
python Eikos-4B-INT4/serve.py --model $PWD/Eikos-4B-INT4 --vllm-url http://127.0.0.1:8001 --port 8000 # HTTP API on :8000
The HTTP API, agent sessions and the question types are the same as for Eikos-4B.
Images (serve v1.3): this build reads images too; its vision tower is kept in bf16. Send them in
"images", as multipart files or inside the state: see Images for the
format, the limits and the full results. Zero-shot, on the same public items for both builds:
| 300 items per set | Eikos-4B (bf16) | Eikos-4B-INT4 |
|---|---|---|
| MME (yes/no) | 89.7% | 87.3% |
| MMStar | 67.0% | 66.7% |
| SEED-Bench-2-Plus | 73.7% | 71.7% |
| ScreenSpot-v2, no marks | 55.7% | 55.0% |
| ScreenSpot-v2, 4×4 grid drawn | 53.3% | 54.0% |
| ScreenSpot-Pro, no marks | 52.7% | 52.0% |
| Same answer as bf16: all / confident (≥0.9) | — | 88.9% / 99.2% |
Images are harder than our text suites: the bf16 model is at least 90% confident on only 21% of these items. Of the 199 answers (out of 1,800) that differ from bf16, 96% are on items where bf16 itself was below 0.7 confidence. The largest gap is on MME (yes/no): 87.3% against 89.7% for bf16.
Validation against bf16
Same 7,371 items for both builds (7 suites, never used in training), vLLM 0.30 with batching and prefix cache on. The release gate was fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.
| Eikos-4B (bf16) | Eikos-4B-INT4 | |
|---|---|---|
| Size | 9.3 GB | 4.0 GB |
| JevBench public — original / hard | 91.7 / 73.9 | 91.7 / 72.1 |
| DecisionBench — medium / hard | 77.1 / 66.6 | 77.1 / 66.2 |
| General battery (9 tasks) | 75.7 | 75.6 |
| Finance (CUAD, sentiment, FinQA-judge) | 74.7 | 75.7 |
| Trade rules — seen / unseen | 74.5 / 76.0 | 74.9 / 76.0 |
| Compositional rules — same type / new domain / rulebooks | 96.0 / 91.5 / 91.7 | 95.8 / 91.7 / 90.7 |
| ECE (lower is better) | 0.033 | 0.033 |
| ≥0.90 confidence: decides / error | 34.7% / 2.5% | 32.9% / 2.0% |
| Same answer as bf16 (all / confident ≥0.9) | — | 95.5% / 100.0% |
It passes our release gate on accuracy and calibration, but not on answer agreement (95.5% of answers are the same as bf16; the gate asks for 97%). Most changed answers (94%) are on items where the bf16 model itself was unsure (confidence below 0.7); on decisions the bf16 model takes with confidence ≥0.9, 100.0% of answers are the same.
License
MIT for our contributions (LICENSE). The base model, Qwen3.5-4B, is Apache-2.0 (LICENSE-Qwen);
attributions are in NOTICE. Not legal, tax or investment advice.