d3-turbo-NVFP4

d3-turbo-NVFP4 is d3-turbo, the upgraded 27B multimodal decision model of Decision 3.0, with its decoder's projections in NVFP4 for NVIDIA Blackwell GPUs. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, rate on a scale, or point to where something is in an image or a video. It reads the input once per request, answers every question on it and returns a probability for every answer, as d3-turbo does, in 17.2 GB of weights instead of 52.2 GB and 1.7× to 2.1× as fast.

Highlights

• 71.06 on the Jev Decision Index 0.3 public suite against d3-turbo's 71.65, every area within 1.1 points (Knowledge & Reasoning the most), with d3-turbo's top answer on 97.7% of 272,229 questions.
• 76.68 on the public vision suite against 76.96, with d3-turbo's top answer on 93.2% of 12,210 questions.
• 1.69× to 2.12× d3-turbo's speed per request on one NVIDIA RTX PRO 6000, depending on the kind of request (text 96 ms against 203 ms at the median), and 1.87× its text throughput under load.
• 17.2 GB of weights instead of 52.2 GB; 49.1 GiB of GPU memory at most in the speed benchmark against 91.0 GiB.
• Box and tracking lose the most: Box accuracy at IoU 0.5 is 0.881 against 0.897, tracking success AUC 0.773 / 0.790 against 0.792 / 0.807. Images in questions and options keep 0.955 against 0.958.

Quickstart

pip install "transformers==5.17.0" "compressed-tensors>=0.19" "torch>=2.11" torchvision triton pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention  # optional: fast GPU kernels for the linear-attention layers

import sys

from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("vllm-sr/d3-turbo-NVFP4")
sys.path.insert(0, path)
from d3q.load import load_d3

model = load_d3(path, device="cuda:0")


def asset(name):
    return hf_hub_download("vllm-sr/d3-turbo", f"assets/{name}")


# Text: several questions, one call
print(model.system_one(
    state="Ticket: the invoice total is wrong after the plan change.",
    questions={
        "route": {"type": "choice", "instructions": "Which team handles this?",
                  "criteria": {"billing": "Billing", "tech": "Technical support"}},
        "urgent": {"type": "noul", "instructions": "Is the customer asking for an urgent fix?"},
    },
))

# Box: where is it in the image? Four questions, one read of the photo
print(model.system_one(
    state="A photo from a dog festival.",
    images=[asset("example-frisbee.jpg")],  # local paths, http(s) URLs, PIL images or base64-encoded images
    questions={
        "frisbee": {"type": "box", "instructions": "The frisbee"},
        "tail": {"type": "box", "instructions": "The dog's tail"},
        "cone": {"type": "box", "instructions": "The orange cone"},
        "camera": {"type": "box", "instructions": "The camera on the tripod"},
    },
))

# Images in options: each option is an image that only this question sees
print(model.system_one(
    state="This animal: <animal>",
    questions={"pick": {"type": "choice", "instructions": "Which picture shows the same kind of animal?",
                        "criteria": {"a": "<a>", "b": "<b>", "c": "<c>"}}},
    assets={"<animal>": asset("example-panda.jpg"), "<a>": asset("example-option-a.jpg"),
            "<b>": asset("example-option-b.jpg"), "<c>": asset("example-option-c.jpg")},
))

# Video: a track over the whole clip, two boxes per second
print(model.system_one(
    state="A bird at a feeder.",
    videos=[asset("example-hummingbird.mp4")],  # local paths, http(s) URLs, base64-encoded videos or frame arrays
    questions={"target": {"type": "track", "instructions": "The hummingbird", "rate": 2}},
))

load_d3 is d3-turbo's own runtime (d3_runtime.py) with its backbone loader swapped for d3q/: transformers loads the checkpoint with its FP4 weights still packed, and each quantized projection becomes a layer that quantizes its input to NVFP4 with one Triton kernel and multiplies on the FP4 tensor cores in torch.nn.functional.scaled_mm. Prompting, reading the input once, the FP32 readout, Box and Track answers, images and videos are d3-turbo's own; the answers are formatted as on d3-turbo's card. AutoModel.from_pretrained(..., trust_remote_code=True) alone dequantizes the whole checkpoint to BF16 and needs about 52 GB of GPU memory again. Checked on one NVIDIA RTX PRO 6000 with the pip lines above (transformers 5.17.0, compressed-tensors 0.19.0, flash-linear-attention 0.6.0): the four requests are answered as the model runtime below answers them (each edge of the frisbee's box within 0.003 of the runtime's, picture b at 0.9993 against 0.9996). Every number in Evaluation comes from the model runtime.

Run with vLLM Semantic Router

The vLLM Semantic Router model runtime loads d3-turbo-NVFP4 as it loads d3-turbo, verifies the package against its MODEL_MANIFEST.json, and keeps the weights in NVFP4 on the GPU, multiplying on the FP4 tensor cores with its own NVFP4 layers (#5225, #5233) on d3-turbo's read-once path (#5275). Every number on this card comes from that runtime:

pip install -e "src/model-runtime[multimodal]" flash-linear-attention   # in a checkout of vllm-project/semantic-router, main at e38f1fe7 or later
vllm-srun serve vllm-sr/d3-turbo-NVFP4 --revision b02436590c867d0cf468dc2b2b60695c263b80a9 --port 8899 --max-request-bytes 67108864

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
  "model": "vllm-sr/d3-turbo-NVFP4",
  "state": "Write a Python function that reverses a string.",
  "questions": {
    "kind": {"type": "choice", "instructions": "What kind of work is this?",
             "criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
    "reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
  }
}'

vllm-srun takes a Hub package that is not built in only at a full 40-hex --revision; requests with videos need --max-request-bytes above the default 8 MiB.

Engine mode serves it behind the same System One API on port 8899, so the request above works unchanged:

curl -fsSL https://vllm-sr.ai/install.sh | bash -s -- --channel dev --no-launch
export PATH="$HOME/.local/bin:$PATH"
vllm-sr serve vllm-sr/d3-turbo-NVFP4 --engine

The first start downloads the 17.2 GB package and the engine image and waits at most 10 minutes for the model to be ready, so on a slow link download the package into the engine's cache first: huggingface-cli download vllm-sr/d3-turbo-NVFP4 --cache-dir ~/.local/state/vllm-sr/engine/vllm-sr--d3-turbo-NVFP4/models/model-runtime. With the model already downloaded, it is ready in about 60 seconds on the RTX PRO 6000 (dev build 0.5.0.dev20261020093359) and answers that request and Box questions as vllm-srun does. vllm-sr stop stops it.

Serving performance

By input length, by number of questions, by area and family

By area the text speed-up is 1.94× to 2.10× (arts 2.03×, knowledge 2.10×, language 1.94×, retrieval 1.96×, tools 1.99×); by image benchmark 1.23× (CV-Bench) to 1.85× (KIE); by Box family 1.43× to 1.88×; video tracking 1.66× at 1 update per second and 1.57× at 2.

Earlier, separate timings on the same GPU: one question per request on the tasksource test split below, one request at a time with exact kernels, median 30.0 ms against 59.2 ms; GPU memory 19.9 GiB after loading, 25.4 GiB at most on that split, 26.2 GiB on the public vision suite (d3-turbo 57.9 GiB), 25.1 GiB on tracking at 2 updates per second, 27.2 GiB with 255 questions about one photo. The Box, images-in-options and tracking tests below, 8 requests at a time (tracking one clip at a time), took 469 s (p50 0.60 s), 216 s (0.62 s), 3.2 min (0.76 s per clip, rate 1) and 6.6 min (2.49 s, rate 2), against 898 s (1.09 s), 395 s (1.14 s), 5.6 min (1.26 s) and 10.8 min (3.91 s) for d3-turbo (59.1 GiB at most). Ask once with assets/example-panda.jpg of d3-turbo, p50 / p90 110 / 111, 125 / 126, 182 / 183, 456 / 457, 875 / 876 ms and 1.83 / 1.83 s for 1 / 4 / 16 / 64 / 128 / 255 questions.

• Throughput under load does not grow past 4 requests at a time for either package with these requests, and latency grows with the queue, probably because the longest requests fill the runtime's token budget per forward on their own (not measured).
• GPU memory is the device's used memory (nvidia-smi), PyTorch's cached blocks included, with requests of up to about 15,000 tokens.
• The benchmark is the fixed set of 5,000 requests in the dataset vllm-sr/d3-perf-bench (private to the vLLM Semantic Router team; text across seven input lengths and the Decision Index's five areas, images, Box, ask once and video), which times any d3-turbo package on the same requests.

_{Measured on 2026-10-22 on one NVIDIA RTX PRO 6000 Blackwell Workstation Edition (driver 595.99.02), PyTorch 2.14.1+cu130, with the vLLM Semantic Router model runtime from main at e38f1fe7 and its default settings (--dtype auto, --max-request-bytes 67108864). Each package was served alone on the GPU, one after the other: every request one at a time, then the text requests 4, 16 and 64 at a time.}

Evaluation

Every comparison is against d3-turbo (BF16) on the same questions. Through the same server, requests and scoring, d3-turbo gives the same hit or miss as its own evaluation rows on 99.5% of the Box items and the same right or wrong answer on 99.7% of the images-in-options items, so the differences below are the quantization's.

Text: Jev Decision Index 0.3, public suite

The 110,087 requests (272,229 questions) that d3-turbo v1.1's own run answered, scored by the official 0.3 kit; the d3-turbo column is that run's results, which d3-turbo's card reports as 71.7. The suite's other 30,091 rows (MMLU, RouterBench, SGD, ARC, SimpleBench) are not part of the 0.3 index. This package gives d3-turbo's top answer on 97.65% of the questions (6,389 change); the lowest agreement is on ChessBench legal moves (78.9%), GPQA-Diamond (81.6%), HLE text (81.6%) and MMLU-Pro chemistry (83.5%). KL(d3-turbo ‖ this package): mean 0.0079, median 0.00029, 95th percentile 0.034, 99th percentile 0.135. The previous weights, calibrated on text only, scored 71.25 (Knowledge 58.6) with 97.68% agreement.

Images: Jev Decision Index 0.3, public vision suite

By benchmark (accuracy)

All 12,210 rows, both packages through the same engine. d3-turbo's card gives its vision board estimate (72.9), which is not this public index. This package gives d3-turbo's top answer on 93.24% of the questions (826 change), most of the changes on R-Bench-M (79.1% agreement) and MMMU-Pro vision (82.7%), where d3-turbo itself is right about half the time; every other benchmark agrees on 93% to 99%. KL: mean 0.027, median 0.0033, 99th percentile 0.327.

Images: Box and images in options

d3-turbo's own BoxSet-test (5,836 items) and ComposeSet-test (2,752 questions), scored as d3-turbo's evaluation scores them. On Box this package hits or misses as d3-turbo does on 97.4% of the items (122 hits become misses, 29 misses become hits): −1.6 points at IoU 0.5 (95% interval −2.0 to −1.2), and at IoU 0.75 0.7443 against 0.7959 (mean IoU 0.7856 against 0.8155). Photos lose most (COCO spatial 0.71 → 0.67, exemplar 0.74 → 0.71, unique 0.83 → 0.81), then receipts (1.00 → 0.98); charts, screens, shapes and tables stay within 0.01. On images in options it is right or wrong as d3-turbo is on 98.6% of the questions (24 right answers become wrong, 16 wrong ones right): −0.3 points (95% interval −0.7 to +0.2).

Videos: tracking

The 96 TrackSet-test clips of at most 60 s (2,690 moments at rate 1, 5,377 at rate 2), sent as their own 4 fps JPEG frames muxed into Matroska without re-encoding. This package hits or misses as d3-turbo does on 97.8% and 97.3% of the moments with a visible target, and calls the target visible or not as d3-turbo does on 98.2% and 97.7%.

Text: tasksource-jev-typed-decisions, test split

Every usable row of the test split of tasksource/tasksource-jev-typed-decisions at revision 28c39125 (a state of at most 6,000 characters, no repeated options): 14,031 requests of one question each, 547 answers changed (96.10%). Both packages ran on the RTX PRO 6000 one request at a time, with exact kernels and no shared context; two more d3-turbo runs (16 concurrent requests at --dtype auto, twice) gave the same 14,031 answers. The text calibration prompts come from this dataset's validation split, so this checks the package on the distribution it was calibrated on; the Jev Decision Index suites above are the independent check.

KL by type, Typed Decisions, and the train rows used to choose the method

KL(d3-turbo ‖ this package) on the tasksource test split, mean / median: Choice 0.01556 / 0.000841, Yes / No 0.00668 / 0.000325, Score 0.01538 / 0.002637.

Typed Decisions (LocalLLaMA/typed-decisions, all test split at d9eda7f3: 400 cases of five questions, one /v1/systemone request per case, scored as the dataset's bench/score_with_jobs.py scores them). The d3-turbo column is its own card's result through the transformers runtime, so the columns differ in engine as well as in precision.

Train rows used to choose the method (on the previous, text-only calibration): 5,000 rows of the train split of tasksource-jev-typed-decisions (28c39125, train shard 0, disjoint from the calibration rows), where the GPTQ rounding was picked over nearest rounding. 187 answers change (96.26%; 79% of the rows are Choice).

_{Built on 2026-10-25. Calibration: 384 requests run through d3-turbo's own runtime as it answers them (read-once attention, image and video tokens included), about 255k tokens: 192 prompts of the validation split of tasksource-jev-typed-decisions (28c39125); 96 still images with multiple-choice questions from d3-turbo's vision training corpora (train or generated splits of memes, charts, infographics, GUI screenshots, photos and image pairs), about 12 per kind; 64 Box items and 8 absent-target items of BoxSet-dev; 12 Perception Test train videos; 12 TrackSet-dev clips, one moment each. None comes from a test split used below. The previous weights (revision 0c42028b) were calibrated on 512 text prompts only, which left image and video activations above the inputs' calibrated range: Box 0.860 and tracking AUC 0.755 / 0.768 then. AWQ: per decoder layer and group of projections that read one input, the scale s = x^a / w^(1-a) is searched over 20 values of a to minimise the layer's output error with that group run as NVFP4 weights times NVFP4 activations, then folded into input_layernorm, post_attention_layernorm and up_proj. GPTQ: each projection's weights are rounded to E2M1 column by column, the error spread over the remaining columns through the inverse Hessian of the calibration inputs (1% dampening, blocks of 128 columns), each 16-value block's E4M3 scale taken from its already-compensated weights. q / k / v, in_proj_qkv / in_proj_z and gate / up share their weight and input global scales, so the runtime's merged products apply. Same format and kernels as vllm-sr/d3-turbo-NVFP4-AWQ, whose weights are rounded to nearest. Evaluation runs: the Decision Index suites with the official 0.3 kit (decision-index f8552a1) through the model runtime in process, one request at a time; Box, images in options, tracking and Typed Decisions through vllm-srun serve at --dtype auto; all on the RTX PRO 6000 above.}

License

Apache-2.0 (LICENSE), as d3-turbo. Built on vllm-sr/d3-turbo.

Citation

@misc{d3turbo_2026,
  title        = {{d3-turbo}: A Multimodal Decision Model that Reads Once},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3-turbo}}
}