JevK5-4B v0.3 GGUF — the text decision model, eleven quantizations

A calibrated text decision model: state and typed questions in, one probability per option out, no generated text. The author's own GGUF ships three quantizations for this size; these are the full range. Pure text, so no projector and no vision tower — it runs on a card where the vision variants cannot.

Files

The author's alibiserikbay/JevK5-GGUF ships two or three quantizations per size. These are the full range, in the same style as bartowski and unsloth:

jevk5_config.json carries the calibration and must travel with the weights.

Run it

llama-server -m jevk5-4b-v0.3-Q4_K_M.gguf \
             --host 0.0.0.0 --port 8080 -ngl 99 -c 4096

One file. No projector, no --no-mmproj-offload: this is a text model.

Decision readout

Read the option logits at the answer slot, divide by the calibration temperature, then softmax:

temperature = A calibrated text decision model: state and typed questions in, one
probability per option out, no generated text. The author's own GGUF
ships three quantizations for this size; these are the full range.
Pure text, so no projector and no vision tower — it runs on a card
where the vision variants cannot.       (jevk5_config.json)

That division is client-side and it is what makes the numbers probabilities rather than orderings.

Which size, and why

Whole family: Imajev GGUF

If you want to build your own

JevK5 is a plain text model — Qwen3.5ForCausalLM, no vision tower. That makes it both easier and more particular than the Imajev build:

python convert_hf_to_gguf.py <merged-dir> --outfile out.gguf --outtype f16 --no-mtp
./llama-quantize out.gguf out-Q4_K_M.gguf Q4_K_M

--no-mtp is required and only defined for the text architecture. The config announces mtp_num_hidden_layers and the checkpoint contains no mtp.* tensors, so the converter counts 33 and exports 32, and the loader then looks for a block that was never written. The same trap as the vision build, opposite direction — there the flag is missing, here it is mandatory.

Two things that are easy to lose:

• llama-quantize loads libcudart, libcublas and libnccl dynamically. Without LD_LIBRARY_PATH it dies with a shared-library error and the script that called it reports success.
• The calibration temperature is not in the GGUF. Read it from jevk5_config.json and divide the option logits yourself.

How this fits the grader picture

Measured on 20 pages with three independent graders:

rule (source vs task)   <-> Imajev-2B     rho = +0,086
rule (source vs task)   <-> JPT-4B         rho = +0,027
text decider (Tev1)     <-> image decider  rho = +0,299
two image deciders      <-> each other     rho = +0,265

No two agree, and the correlations were computed on the same twenty pages, not on separate runs. There is no best grader, so the pipeline composes them multiplicatively and takes a consensus rather than picking a winner.

This model is text-only, which is where it earns its place: the rule grader and a text decider both read the source, while a vision decider reads the rendered page. Three different questions, three different instruments.

The 2B gap

alibiserikbay/JevK5-GGUF ships a 2B v0.2 Q8_0 and nothing in v0.3. alibiserikbay/JevK5-2B is safetensors only, and no GGUF of it exists anywhere. If you need a text decision model that fits a 4 GB card, that one is the gap.

Licence and provenance

Built from the Apache-2.0 weights of alibiserikbay/JevK5 (4B, v0.3) or alibiserikbay/JevK5-9B (9B, v0.3.3), on Qwen/Qwen3.5-4B, via llama.cpp convert_hf_to_gguf.py.

The two versions are not the same model under different names: 4B v0.3 is calibrated at T = 1.22, 9B v0.3.3 at T = 1.316. What else changed between them we did not measure, and it is not claimed here.