jev-vlm-4b-lite (GGUF)

A calibrated visual decision model in a single GGUF: give it an image (a document, receipt, chart, or annotated screenshot) and typed questions, and it returns a well-calibrated answer with confidence — not just a guess. It answers three primitives:

• Choice — pick one of N options → probability per option + confidence
• Noul — "is this statement true?" → calibrated P(true)
• Score — rate on a 2–10 rubric → expected level

Distilled (forward-KL) from a pointer-head teacher, so a plain letter-logit GGUF carries pointer-quality calibration. 4B params, Apache-2.0, runs in llama.cpp / LM Studio.

Quick start (llama.cpp)

# 1. grab a quant + the vision projector (mmproj is REQUIRED for images)
hf download sidman007/jev-vlm-4b-lite-gguf jev-vlm-4b-lite-Q8_0.gguf mmproj-jev-vlm-4b-lite-f16.gguf --local-dir .

# 2. serve. --jinja uses the embedded decision template (plain-text answers, no <think>).
#    --image-min-tokens 1024 is REQUIRED — Qwen-VL needs it to read fine text/marks.
llama-server -m jev-vlm-4b-lite-Q8_0.gguf --mmproj mmproj-jev-vlm-4b-lite-f16.gguf \
             --image-min-tokens 1024 --jinja

Get a plain-text answer — send the image + question/options as the user message; the reply is the option letter:

curl -s localhost:8080/v1/chat/completions -d '{
  "messages": [{"role":"user","content":[
    {"type":"image_url","image_url":{"url":"data:image/png;base64,<...>"}},
    {"type":"text","text":"Question: Which value is the total?\nOptions:\nA. 1,240.00\nB. 1,420.00\nC. Not shown"}]}],
  "max_tokens": 1, "temperature": 0
}' | jq -r ".choices[0].message.content"      # -> "B"

Get the calibrated distribution (the point of this model) — same request with logprobs, then softmax the option-letter probabilities:

curl -s localhost:8080/v1/chat/completions -d '{ ...same messages...,
  "max_tokens": 1, "temperature": 0, "logprobs": true, "top_logprobs": 20
}' | jq ".choices[0].logprobs.content[0].top_logprobs"   # softmax over A/B/C = the distribution

• Noul = a 2-option Choice A. false / B. true; P(true) = softmax weight on B.
• Score = an N-level Choice over rubric levels; expected level = Σ i·P(level_i).
• A low top-probability / flat distribution is the model abstaining, not guessing.

LM Studio: import the .gguf + the mmproj, attach an image, and use the chat endpoint — the decision template is embedded, so you get the plain letter. Enable "return token probabilities" for the distribution.

Benchmarks

Measured on the shipped Q8_0 GGUF via the correct serving path (chat + --image-min-tokens 1024); all calibration numbers are on the softmaxed distribution.

The in-domain locked test matches the bf16 model (Q8→Q4 quantization costs ~nothing). AI2D 0.835 is competitive with good ~7B VLMs — from a 4B model reading constrained option-letter logprobs, so it's a comparable data point, not a like-for-like leaderboard submission (the model is document/chart-specialized). MM-UPD abstention beats the always-answer baseline (0) but still confidently answers some unanswerable questions — an honest OOD limitation.

Files & quants

Serving notes (important)

1. Images must go through the chat endpoint. llama.cpp's /completion endpoint silently ignores images (accuracy collapses ~0.95 → ~0.55). Always use /v1/chat/completions with image_url.
2. --image-min-tokens 1024 — required for grounding / fine text.
3. Concurrency: with --parallel N, context is split across slots — set -c (8192 * N) or image requests fail with "exceeds context size" (an image is 1k–4k tokens).
4. Ollama can't import third-party Qwen3.5 GGUFs with mmproj; use llama.cpp or LM Studio.

The best-calibrated variant is the transformers pointer model sidman007/jev-vlm-4b (AutoModel(..., trust_remote_code=True).decide(...)). Converted with --no-nextn.

Open weights under Apache-2.0 — free for commercial and non-commercial use.