4-bit Walsh–Hadamard + Cholesky GPTQ (W4A16-HGPTQ, group size 64) quantization of vllm-sr/d3-nano (vllm-sr/d3 code-readout-v1 single-pass decision model). It compresses the 4.12 GiB BF16 checkpoint (readout.safetensors + backbone) to 1.80 GiB (2.29× smaller) while preserving 100.00% top-1 decision agreement and 0.999635 readout logit cosine similarity against the BF16 reference.
1.80 GiB total across 150 quantized text linears, plus 98 vision tower & merger linears, plus the exact BF16 readout.safetensors (255 × 2048 code-readout head).H_64) + MSE-initialized Cholesky GPTQ (group_size=64, actorder="static", dampening_frac=0.01), calibrated layer-by-layer under noncausal_full_attention attention.24 layers (hidden_size=2048). Single-pass hybrid GatedDeltaNet + full-attention forward pass.model.language_model.* (and model.visual.*) prefixes with "quantization_status": "compressed", loading cleanly with missing: 0, unexpected: 0 in Qwen3_5Model.from_pretrained, D3.from_pretrained, and warp-34.vllm-sr/d3-nano)Single-pass code-readout-v1 evaluation across multi-question choice, yes_no, multi_select, and scale decision tasks (with forward + reverse option-permutation averaging):
| Evaluation | BF16 Base (vllm-sr/d3-nano) |
d3-nano-W4A16-HGPTQ |
Notes |
|---|---|---|---|
Safetensors Size (incl. readout.safetensors) |
4.12 GiB (4,428,644,200 B) |
1.80 GiB (1,933,757,440 B) |
2.29× compression (−2.32 GiB) |
Decision Top-1 Agreement (12 questions) ↑ |
100.00% |
100.00% |
Forward + reverse permutation-averaged readout |
| Option Probability KL Divergence (mean / max nats) ↓ | 0.000000 |
0.000222 / 0.001307 |
Per-question normalized option distribution KL |
| 255-Code Readout Logit Cosine Similarity ↑ | 1.000000 |
0.999635 |
Full 255-code softmax KL 0.000328 nats |
Vision Tower (vision-probe-v2) (96 images, 50,596 visual tokens) |
1.000000 cos sim |
0.961678 mean cos (0.916555 min) |
vision-guard-v2 mean cos 0.965764 (min 0.939876) |
248 linear layers in W4A16-HGPTQ):150 linears): All 24 hybrid GatedDeltaNet + full-attention blocks (q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z, out_proj, gate_proj, up_proj, down_proj) with online block-Hadamard rotation (head_dim=64) + Cholesky GPTQ (group_size=64, actorder="static").98 linears): All 24 ViT blocks (visual.blocks.*.{attn.qkv,attn.proj,mlp.linear_fc1,mlp.linear_fc2}) plus visual.merger.{linear_fc1,linear_fc2} in g64-h64.readout.safetensors (weight [255, 2048]), embed_tokens, linear_attn.in_proj_a, linear_attn.in_proj_b, norms, biases, and visual.patch_embed.proj.from d3_runtime import D3
model = D3.from_pretrained("nchapman/d3-nano-W4A16-HGPTQ", device="cuda:0")
result = model.system_one(
state="Customer ordered item #4412, delivered on time, requested an invoice copy.",
questions={
"intent": {
"type": "choice",
"instructions": "What does the customer need?",
"criteria": {"invoice": "Copy of invoice", "refund": "Return and refund"},
}
},
)
print(result)
model.safetensors 1.9 GBreadout.safetensors 2 MBvllm-sr/d3-nano