Ornith-1.5-35B-A3B-DFlash2-2.6B-A0.3B

A Mixture-of-Experts DFlash2 drafter for ornith-ai/Ornith-1.5-35B-A3B with a trained confidence head.

Results

Accepted length is calculated from the raw serving counters:

accepted_length = 1 + accepted_draft_tokens / draft_rounds

Architecture

Example: Run with vLLM

This is a drafter checkpoint, not a standalone language model. Pair it with Ornith-1.5-35B-A3B or a compatible quantized version:

vllm serve ornith-ai/Ornith-1.5-35B-A3B \
  --tensor-parallel-size 1 \
  --enable-prefix-caching \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3 \
  --speculative-config '{
    "method": "dflash",
    "model": "DaoCloud/Ornith-1.5-35B-A3B-DFlash2-2.6B-A0.3B",
    "num_speculative_tokens": 7
  }'

The checkpoint currently requires a vLLM build containing:

• vllm-project/vllm#54154 — sample_from_anchor=true query layouts for DFlash draft models.
• vllm-project/vllm#54475 — MoE DFlash2 draft models.
• vllm-project/vllm#55668 — optional DFlash2 confidence-head loading.

Checkpoint

The BF16 checkpoint contains 65 tensors and 2,638,552,065 parameters.

Future work

• Adaptive verification: confidence-head-driven adaptive draft budgets for high-concurrency serving, with calibration data. This will need variable-length GDN decode from vllm-project/vllm#53929. The fixed budget used here never produces those ragged layouts, which is why the three pull requests above are sufficient today.

• NVFP4 quantized weights.

• Throughput and latency Pareto curves across concurrency levels.

Acknowledgements

This work builds on ornith-ai/Ornith-1.5-35B-A3B, vllm-project/speculators, and vllm-project/vllm.

License

Apache-2.0.