d3-ultra-FP8

d3-ultra-FP8 is d3-ultra, the newest 27B multimodal decision model of Decision 3.0, with its decoder's projections in FP8. Give it an input (text or JSON, optionally with images and videos) and the questions you need answered: pick one of several options, say yes or no, rate on a scale, or point to where something is in an image or a video. It reads the input once per request, answers every question on it and returns a probability for every answer, as d3-ultra does, in 31.9 GB of weights instead of 52.2 GB.

Highlights

• 71.52 on the Jev Decision Index 0.3 public suite against d3-ultra's 71.57, every area within 0.3 points, with d3-ultra's top answer on 99.3% of 272,229 questions.
• 75.51 on the public vision suite against 75.35 for d3-ultra on the same GPU, with d3-ultra's top answer on 97.7% of 12,210 questions.
• Box and tracking hold: Box accuracy at IoU 0.5 is 0.871 against 0.870, tracking success AUC 0.774 / 0.780 against 0.778 / 0.782, and images in questions and options 0.952 against 0.952.
• 31.9 GB of weights instead of 52.2 GB, and 40 ms against 51 ms at the median for one text question at a time on one AMD Instinct MI300X.

Quickstart

pip install "transformers==5.17.0" "compressed-tensors>=0.17" torch torchvision triton pillow opencv-python-headless safetensors accelerate
pip install flash-linear-attention  # optional: fast GPU kernels for the linear-attention layers

import sys

from huggingface_hub import hf_hub_download, snapshot_download

path = snapshot_download("vllm-sr/d3-ultra-FP8")
sys.path.insert(0, path)
from d3q.load import load_d3

model = load_d3(path, device="cuda:0")


def asset(name):
    return hf_hub_download("vllm-sr/d3-ultra", f"assets/{name}")


# Text: several questions, one call
print(model.system_one(
    state="Ticket: the invoice total is wrong after the plan change.",
    questions={
        "route": {"type": "choice", "instructions": "Which team handles this?",
                  "criteria": {"billing": "Billing", "tech": "Technical support"}},
        "urgent": {"type": "noul", "instructions": "Is the customer asking for an urgent fix?"},
    },
))

# Box: who is in the canoe? Three questions, one read of the painting
print(model.system_one(
    state="A painting of fur traders on a river.",
    images=[asset("fur-traders.jpg")],  # local paths, http(s) URLs, PIL images or base64-encoded images
    questions={
        "cat": {"type": "box", "instructions": "The cat"},
        "canoe": {"type": "box", "instructions": "The canoe"},
        "dog": {"type": "box", "instructions": "A dog"},
    },
))

# Photo to etching: the question holds a photo, each option is an etching
print(model.system_one(
    state="An architecture quiz.",
    questions={"pick": {"type": "choice", "instructions": "Which 18th-century etching shows the same building as <building>?",
                        "criteria": {"a": "<a>", "b": "<b>", "c": "<c>", "d": "<d>"}}},
    assets={"<building>": asset("pantheon.jpg"), "<a>": asset("pantheon-a.jpg"), "<b>": asset("pantheon-b.jpg"),
            "<c>": asset("pantheon-c.jpg"), "<d>": asset("pantheon-d.jpg")},
))

load_d3 is d3-ultra's own runtime (d3_runtime.py) with its backbone loader swapped for d3q/: transformers loads the checkpoint with its weights in FP8, and each quantized projection becomes a layer that quantizes its input to FP8 per row with one Triton kernel and multiplies in torch._scaled_mm. Prompting, reading the input once, the FP32 readout, Box and Track answers, images and videos are d3-ultra's own; the answers are formatted as on d3-ultra's card. AutoModel.from_pretrained(..., trust_remote_code=True) alone does not run FP8 weights.

Checked on one AMD Instinct MI300X with transformers 5.17.0 and compressed-tensors 0.17.0: the three requests above are answered as d3-ultra answers them through the same path. Every edge of the cat and canoe boxes is within 0.004 of d3-ultra's, the dog is not found by either (confidence 0.05 against 0.00), etching d gets 0.996 from both, and the ticket goes to billing with the same probability to five decimals.

Run with vLLM Semantic Router

The vLLM Semantic Router model runtime loads d3-ultra-FP8 as it loads d3-ultra, verifies the package against its MODEL_MANIFEST.json, and keeps the weights in FP8 on the GPU:

pip install -e "src/model-runtime[multimodal]" flash-linear-attention   # in a checkout of vllm-project/semantic-router
vllm-srun serve vllm-sr/d3-ultra-FP8 --revision 440be8e8509f157f2cb61950ea412bb3436a4ea6 --port 8899 --max-request-bytes 67108864

curl -s localhost:8899/v1/systemone -H 'content-type: application/json' -d '{
  "model": "vllm-sr/d3-ultra-FP8",
  "state": "Write a Python function that reverses a string.",
  "questions": {
    "kind": {"type": "choice", "instructions": "What kind of work is this?",
             "criteria": {"code": "Writing or fixing code", "math": "Mathematics", "chat": "Anything else"}},
    "reasoning": {"type": "noul", "instructions": "Does answering this need multi-step reasoning?"}
  }
}'

This package was served with the model runtime from main as of 2026-10-29. A runtime that predates d3-ultra's prompt layout and per-request temperature refuses the package, since the model identity in its manifest differs. vllm-srun takes a Hub package that is not built in only at a full 40-hex --revision; requests with videos need --max-request-bytes above the default 8 MiB. The Box, images-in-options, tracking and tasksource results below come from the model runtime run on a local copy of this repository (the same weights and loaded files).

The full speed benchmark of the d3-turbo packages has not been run on this package. One question per request on the tasksource test split below, one request at a time through the model runtime on one AMD Instinct MI300X: median 40.4 ms against 51.3 ms for d3-ultra.

Evaluation

Every comparison is against d3-ultra (BF16) on the same questions. The text index compares with d3-ultra's own public run (vllm-sr/d3-ultra-decision-index); images, Box, images in options and tracking compare with d3-ultra run on the same MI300X through the same engine. d3-ultra's vision run on this MI300X gives 75.35 against 75.33 in its public run, so the differences below are the quantization's.

Text: Jev Decision Index 0.3, public suite

The 140,178 requests of d3-ultra's public run, scored by the official 0.3 kit; the index reads the 110,087 requests (272,229 questions) of its 37 benchmarks. Re-scored by the same scorer as this package, d3-ultra's results give the 71.57 of its public run. This package's run, on another build of the same 0.3 suite, also answered 442 requests that d3-ultra's run does not have; they are left out, and scoring this package on all its own requests gives the same 71.52. This package gives d3-ultra's top answer on 99.26% of the index's questions (2,023 change); the lowest agreement is on HLE text (92.0%), ChessBench legal moves (94.8%) and GPQA Diamond (94.9%). KL(d3-ultra ‖ this package): mean 0.0012, median 0.000022, 95th percentile 0.0043, 99th percentile 0.022.

Images: Jev Decision Index 0.3, public vision suite

By benchmark (accuracy)

All 12,210 rows, both packages through the same engine on the same MI300X. This package gives d3-ultra's top answer on 97.72% of the questions (278 change), most of the changes on R-Bench-M (91.9% agreement) and MMMU-Pro vision (93.0%), where d3-ultra itself is right about half the time; every other benchmark agrees on 97.9% to 99.8%. KL: mean 0.0036, median 0.00018, 99th percentile 0.051.

Images: Box and images in options

The Box test set of d3-turbo (5,836 items) and its images-in-options test set (2,752 questions), scored as d3-turbo's evaluation scores them. On Box this package hits or misses as d3-ultra does on 98.9% of the items (32 hits become misses, 34 misses become hits): +0.03 points at IoU 0.5 (95% interval −0.2 to +0.3); at IoU 0.75 it is 1.2 points lower (mean IoU 0.766 against 0.771). Every family stays within 0.011 (COCO exemplar 0.679 → 0.688, spatial 0.650 → 0.655, unique 0.799 → 0.802, receipts 0.957 → 0.946; charts, screens, shapes and tables within 0.002). On images in options it is right or wrong as d3-ultra is on 99.5% of the questions (7 right answers become wrong, 7 wrong ones right).

Videos: tracking

The 96 tracking test clips of d3-turbo of at most 60 s (2,690 moments at rate 1, 5,377 at rate 2), sent as their own JPEG frames muxed into Matroska at the clip's frame rate without re-encoding. At rate 1, the tracks of the three targets of one clip (120 moments) come back with times that do not line up with the test's moments for both packages and count as misses. This package hits or misses as d3-ultra does on 99.1% and 99.2% of the moments with a visible target, and calls the target visible or not as d3-ultra does on 99.3% and 98.8%.

Text: tasksource-jev-typed-decisions, test split

Every usable row of the test split of tasksource/tasksource-jev-typed-decisions (a state of at most 6,000 characters, no repeated options): 14,031 requests of one question each, answered by the vLLM Semantic Router model runtime on one AMD Instinct MI300X. Agreement compares the top answer of every question with d3-ultra on the same requests (Choice and Score: the highest-probability option; Yes / No: p ≥ 0.5). An answer is correct when it is the request's target (Yes / No: the target's side of 0.5). Accuracy is given per type only, since the three types count a correct answer differently. KL(d3-ultra ‖ this package), mean / median: Choice 0.0046 / 0.000021, Yes / No 0.0021 / 0.000016, Score 0.0045 / 0.00021.

_{Built on 2026-10-29 by fp8_package.py in bench/d3-quant-eval, the recipe of vllm-sr/d3-turbo-FP8: each output channel's scale is its largest absolute weight divided by 448, rounded to BF16. Evaluation runs, all on one AMD Instinct MI300X: the Decision Index text suite with the official 0.3 kit through d3q, one request at a time; the public vision suite with the same kit; Box, images in options, tracking and tasksource through the model runtime from main as of 2026-10-29 at its default settings.}

License

Apache-2.0 (LICENSE), as d3-ultra. Built on vllm-sr/d3-ultra.

Citation

@misc{d3ultra_2026,
  title        = {{d3-ultra}: A Multimodal Decision Model},
  author       = {{vLLM Semantic Router Team}},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/vllm-sr/d3-ultra}}
}