d3-nano is a 2.21B-parameter multimodal decision model from the vLLM Semantic Router team, including a 0.33B vision encoder. Built on Qwen3.5-2B and released under Apache-2.0, it accepts text or JSON, optionally with images and videos, and answers multiple Choice, Yes/No, or Score questions in one call, returning probabilities without generating text. It scores 38.9 on the official Jev Decision Index 0.3.2 text board—first among similarly sized entries and on the Pareto frontier—and 54.7 on the vision board; its Perception Test video score is 63.5. On one AMD Instinct MI325X, median latency is 13.4 ms for text, 90.2 ms for an image request, and 301 ms for a 10-second video; vLLM Semantic Router with batching serves up to 318 one-question text requests per second. It can be used with Hugging Face Transformers or served through vLLM Semantic Router Engine mode.
[!NOTE] This is a Decision Model GGUF that supports the llama.cpp
v1/systemoneAPI endpoint.
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| d3-nano.BF16.gguf | BF16 | 4.79 GB | Link | Full BF16 weights. Highest quality, largest file size. |
| d3-nano.Q3_K_L.gguf | Q3_K_L | 1.38 GB | Link | Lower quality but usable, good for low RAM availability. |
| d3-nano.Q3_K_M.gguf | Q3_K_M | 1.32 GB | Link | Low quality. |
| d3-nano.Q4_K_M.gguf | Q4_K_M | 1.56 GB | Link | Good quality, default size for most use cases, recommended. |
| d3-nano.Q4_K_S.gguf | Q4_K_S | 1.5 GB | Link | Slightly lower quality with more space savings, recommended. |
| d3-nano.Q5_K_M.gguf | Q5_K_M | 1.76 GB | Link | High quality, recommended. |
| d3-nano.Q5_K_S.gguf | Q5_K_S | 1.72 GB | Link | High quality, recommended. |
| d3-nano.Q6_K.gguf | Q6_K | 1.97 GB | Link | Very high quality, near perfect, recommended. |
| d3-nano.Q8_0.gguf | Q8_0 | 2.55 GB | Link | Extremely high quality, generally unneeded but max available quant. |
| d3-nano.mmproj-bf16.gguf | mmproj-bf16 | 671 MB | Link | Multimodal projection file in BF16 format. Used for vision/language models. |
Launch llama-server with the GGUF model and wait for the health check endpoint to report ready.
import subprocess, time, requests
MODEL_PATH = "/data/llama.cpp/gguf-new/d3-nano.BF16.gguf"
BASE_URL = "http://127.0.0.1:8080"
server = subprocess.Popen(
[
"/data/llama.cpp/build/bin/llama-server",
"-m", MODEL_PATH,
"--host", "127.0.0.1",
"--port", "8080",
"-ngl", "99",
],
stdout=open("/data/llama-server.log", "w"),
stderr=subprocess.STDOUT,
)
for _ in range(120):
try:
if requests.get(f"{BASE_URL}/health", timeout=2).status_code == 200:
print("server ready")
break
except requests.exceptions.RequestException:
pass
time.sleep(2)
else:
print("server not ready, check /data/llama-server.log")
/v1/systemone EndpointSend a POST request with a state context and structured questions (choice, noul, or score).
curl -s http://127.0.0.1:8080/v1/systemone \
-H "Content-Type: application/json" \
-d '{
"state": "Customer message: I was charged twice for my order last week and nobody has replied.",
"questions": {
"route": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "payments, charges, refunds, invoices",
"shipping": "delivery, tracking, lost or late parcels",
"technical": "bugs, errors, login problems"
}
},
"angry": {
"type": "noul",
"instructions": "Is the customer angry?"
},
"urgency": {
"type": "score",
"instructions": "How urgent is this?",
"criteria": ["can wait", "this week", "today", "right now"]
}
}
}' | python3 -m json.tool
{
"model": "/data/llama.cpp/gguf-new/d3-nano.BF16.gguf",
"answers": {
"route": {
"type": "choice",
"choice": "billing",
"probabilities": {
"billing": 0.9996550794867408,
"shipping": 0.0001900160453319817,
"technical": 0.00015490446792721938
},
"confidence": 0.9969671505983558
},
"angry": {
"type": "noul",
"noul": 0.9746957167505704
},
"urgency": {
"type": "score",
"score": 2.4617961533870094,
"legend": {
"0": "can wait",
"1": "this week",
"2": "today",
"3": "right now"
},
"probabilities": {
"0": 0.06293201172016659,
"1": 0.11729182460596137,
"2": 0.11482416224056817,
"3": 0.7049520014333038
},
"confidence": 0.3360664431382385
}
},
"usage": {
"input_tokens": 350,
"output_tokens": 0
}
}
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
Quantized using llama.cpp by prithivMLmods.
vllm-sr/d3-nano