Mizan v3 · 3B · GGUF

by Nimruz · main card and full guide: Mizan-v3-3B · all models: Mizan collection

GGUF builds of Mizan v3, a Persian-first decision model. It answers typed questions (choice, score, yes/no) about a record with calibrated probabilities, in one forward pass, and it follows the policies and rubrics you put in the record. The files carry the decision head and the calibrated temperatures, and llama.cpp serves them through its Jev/SystemOne-compatible /v1/systemone endpoint.

• Q4 is the default: it runs on any 8 GB GPU or a laptop CPU.
• Q8 is about 0.6 points more accurate.
• Text only: there is no vision projector. For images, use Mizan-v2-4B-GGUF.

Run

Needs llama.cpp b11443 or newer (brew install llama.cpp, or a release build).

hf download Nimruz/Mizan-v3-3B-GGUF mizan-v3-3b-Q4_K_M.gguf --local-dir .
llama-server -m mizan-v3-3b-Q4_K_M.gguf -c 8192 -b 8192 -ub 8192 -ngl 99 --port 8080   # -ngl 0 for CPU

curl http://localhost:8080/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"message": "سلام، از دیروز اینترنت کل ساختمون ما قطعه، حدود ۴۰ واحد. کد پیگیری K48213",
            "policy": "ارجاع: قطعی و کندی ← شبکه؛ قبض ← صورتحساب."},
  "questions": {
    "route": {"type": "choice", "instructions": "طبق قواعد، این مورد به کدام تیم برود؟",
              "criteria": {"network": "شبکه", "billing": "صورتحساب", "sales": "فروش"}},
    "has_reference": {"type": "noul", "instructions": "کاربر شماره پیگیری داده است."}}}'

• -ub must cover your longest input, because a decision reads the whole input in one batch.
• Median latency is about 55 ms per text request on an NVIDIA L4, 1.5× faster than Mizan 1.0.

The step-by-step guide, how to read answers, question-writing tips, all three benchmarks and the training details are on the main model card. Older GGUFs (1.0, 1.5) are kept as the v1 and v1.5 tags of this repo.

License

Apache-2.0.