Mizan v2 · 4B · GGUF
▶ Try it live: Mizan demo Space · by Nimruz
Which Mizan?
- Images (camera frames, receipts, product photos): this model, Mizan v2.
- Text only: use Mizan v3 3B. It's smaller (1.9 GB in 4-bit), 1.5× faster, and better at following your policies and scoring rubrics.
- All models: Mizan collection.
GGUF builds of Mizan v2 4B, a Persian-first decision model
for text and images. It answers typed questions (choice, score, yes/no) with calibrated probabilities, in one
forward pass, through llama.cpp's Jev/SystemOne-compatible /v1/systemone.
| File | What | Size | Text (real Persian benchmark) |
|---|---|---|---|
mizan-v2-4b-Q8_0.gguf |
8-bit model | 4.5 GB | 82.8% |
mizan-v2-4b-Q4_K_M.gguf |
4-bit model | 2.7 GB | 82.7% |
mmproj-mizan-v2-4b-Q8_0.gguf |
vision projector, needed for images only | 0.37 GB | – |
The full-precision model scores 82.8% on text.
Images through llama.cpp (Q4 + projector, on images never seen in training, balanced accuracy):
| Question | Balanced accuracy | n |
|---|---|---|
| Person in frame? | 95% | 51 |
| Vehicle in frame? | 100% | 42 |
| Vehicle type | 87% | 44 |
| Person in red zone? | 70% | 12 |
| Vehicle in red zone? | 92% | 16 |
| How many people | 92% | 40 |
| Receipt total above X? | 97% | 150 |
| Receipt line items | 95% | 91 |
| Which product (factory) | 100% | 70 |
| Product defective? | 77% | 150 |
The samples are small (12–150 per question), so expect ±5–10 points; the main card has the larger full-model test.
Run
Needs llama.cpp b11443 or newer.
huggingface-cli download Nimruz/Mizan-v2-4B-GGUF mizan-v2-4b-Q4_K_M.gguf mmproj-mizan-v2-4b-Q8_0.gguf --local-dir .
# text + images (drop --mmproj for text only)
llama-server -m mizan-v2-4b-Q4_K_M.gguf --mmproj mmproj-mizan-v2-4b-Q8_0.gguf -c 8192 -b 8192 -ub 8192 -ngl 99 --port 8080
Send images as data URLs in an images list next to state and questions:
IMG=$(base64 -i cam03.jpg | tr -d '\n')
curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
"state": {"منبع": "دوربین پارکینگ"},
"images": ["data:image/jpeg;base64,'"$IMG"'"],
"questions": {"person_in_zone": {"type": "noul", "instructions": "آیا کسی داخل کادر قرمز هست؟"}}}'
-ubmust cover your longest input, including image tokens.- For real-time use, run it on a GPU. An entry-level 8 GB card is enough.
- CPU works for batch jobs (about 3 s per text request on 16 cores).
The full benchmarks, what it can't do yet (defect location, real CCTV caveats) and training details are on the main model card.
License
Apache-2.0.