Agens Volundr 32B v1 — DFlash2 drafter

Main model card, benchmarks and discussion: Blockway/Agens-Volundr-32B-v1

A DFlash2 block drafter for speculative decoding with Agens Volundr 32B v1. At each step it drafts 8 tokens at once from the target model's hidden states (taps at layers 8, 21, 35, 53 and 67), and the target verifies them in one pass. The target checks every token, so answers keep the target model's quality; with greedy decoding the output can differ from non-speculative decoding only where two tokens are numerically tied.

The drafter has 5 layers (about 1.9B parameters, 3.8 GB (3.6 GiB) in bf16) and shares the target's embeddings and output head, which are not stored here. It is trained on the v1 weights; the Preview drafter is not compatible with v1.

Speed-up

Single user, greedy, 400-token replies, the Blockway sglang build (v1 images), same server with and without the drafter:

The drafter is for one or two concurrent streams. It and its speculative state take memory from the KV cache: on a 48 GB pair with 4 request slots about 28K tokens of KV pool remain (prompts of 30K tokens are rejected), against 74K tokens at one slot. With 8 or more concurrent users plain decoding gives more total throughput; for multi-turn session serving use the no-drafter configurations in the serving README.

Use

Requires the Blockway sglang build (ghcr.io/blockwayz/agens-sglang:v1-sm89 / :v1-sm90, source at github.com/BlockWayz/agens-sglang). Add these flags to the two-GPU bf16 command from the model card, and run the container with -e PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (the v1 images default to a 256 MB FlashInfer workspace; on the 2026-10-05 preview images add -e VOLUNDR_FI_WORKSPACE_MB=256):

    --speculative-algorithm DFLASH \
    --speculative-draft-model-path Blockway/Agens-Volundr-32B-v1-DFlash2 \
    --speculative-num-draft-tokens 8 \
    --speculative-draft-model-quantization unquant \
    --mem-fraction-static 0.90 --context-length 16384 --chunked-prefill-size 2048 \
    --max-mamba-cache-size 4 --max-running-requests 4 --cuda-graph-max-bs 4

Speculative decoding needs --attention-backend flashinfer and serves text requests (leave out --enable-multimodal). With session continuation, use one request slot (--max-mamba-cache-size 3 --max-running-requests 1 --cuda-graph-max-bs 1); see the serving README's recommended configurations.

Licence

Apache-2.0. The licence files and NOTICE ship with the weights.