Agens Volundr 32B v1 — MTP head

Main model card, benchmarks and discussion: Blockway/Agens-Volundr-32B-v1

A multi-token-prediction (MTP) head for speculative decoding with Agens Volundr 32B v1 in Agens Engine. It is one decoder layer (0.42B parameters, 0.85 GB in bf16) that drafts the next tokens from the target model's own last hidden state. It shares the target's embeddings and output layer, which are not stored here.

Current head: v1.1 (2026-10-23), self-distilled. It replaces the v1.0 head of the first release; v1.0 stays downloadable under the tag v1.0. Same inputs, shapes and engine flags, so nothing changes in how you run it.

What changed in v1.1

• Warm-started from v1.0 and trained on 145K samples generated by Agens Volundr 32B v1 itself (chat, code, tool use, agent sessions, Cantonese; thinking on), so the head drafts the model's own distribution rather than a teacher's.
• Draft chain trained to 4 steps (v1.0: 3). The engine's draft order and step counts are unchanged.
• It is a draft source only: the target verifies every drafted token, so greedy output is identical with v1.0, v1.1 or no head (engine gate: 13/13 prompts in one-conversation mode, 48/48 in batched speculation at 48 sequences).

Speed-up, v1.1 over v1.0

Agens Engine 1.3.5, two 141 GB GPUs (compute capability 9.0), --tp 2, Agens Volundr 32B v1 INT4, server mode, one request, greedy, decode tok/s:

From 4 concurrent requests up the two heads give the same throughput. The consumer-profile tables below were measured with the v1.0 head on Engine 1.2 and have not been re-measured with v1.1.

Requirements and limits

• Runs only with Agens Engine 1.2 or later (1.3 or later for images and tensor parallelism), via --mtp DIR (or scripts/agens-run.sh --mtp DIR; on the image, -v DIR:/mtp:ro --mtp /mtp).
• The file is bf16. The engine quantizes the head's linear layers to 4 bits at load (0.22 GB on the GPU, plus 0.07 GB of context buffers).
• Engine 1.3 or later: works with image input and under tensor parallelism; at tp2 / tp4 the engine commits 3.45 / 3.42 tokens per verify step with the head, against 3.10 without it.
• It does not work with the Agens SGLang build or with llama.cpp.

How it drafts

In each speculative step the engine verifies 8 positions. The draft sources fill them in this order: copy from the context fills the slots it covers, this MTP head chains the next ones (up to 4 when the DFlash2 drafter is loaded, all 7 without it), and the DFlash2 drafter fills the rest. Each MTP step scores a 32K-token shortlist of the output layer plus the drafter's candidates for that slot. The target verifies every drafted token, so the drafts change speed, not output.

Speed-up on consumer profiles (v1.0 head, Engine 1.2)

Agens Engine 1.2, one stream, decode tok/s with the MTP head vs without it. The DFlash2 drafter is loaded in both columns, so the speed-up is what the MTP head adds on top of it. Measured on one 48 GB GPU held by the engine to the profile's budget (16 GiB cap for 16gb, 23 GiB for 24gb), short prompts (5 chat, 5 code, 6 thinking, 2 tool-call, 4 agent-session prompts with 8K of context), one run per cell. Sampled cells, and the classes with 2 to 4 prompts, carry run-to-run noise.

16gb profile (Engine-16GB build, images off)

24gb profile (v1 INT4 weights)

The two sampled losses at 24 GB are classes with 2 and 4 prompts.

Acceptance

Tokens committed per verify step, with MTP vs without (same runs):

Context cost

The head and its buffers come out of the KV cache budget. On the 48 GB GPU held to 16 GiB, the 16gb profile serves up to 237,572 tokens of context per conversation with --mtp, against 258,052 without it. The engine plans the KV pool for the GPU it runs on and logs the context it serves. With images on and --mtp, the 16gb profile serves 217,092 tokens and the 24gb profile 258,052 (Engine 1.3); with images off the 16gb profile serves 237,572 and the 24gb profile keeps its full context.

Output is unchanged

• Greedy: on the engine's 13 test prompts, speculative decoding with the MTP head (alone, and with the DFlash2 drafter) gives tokens identical to plain non-speculative greedy decoding: 13/13 in one-conversation mode at 46 GiB and at the 16 GiB 16gb budget, and 13/13 in batched speculation at 2, 4, 8 and 16 sequences. In server mode the replies are identical with and without drafting at 1 to 16 concurrent requests.
• Sampled: 6 prompts (chat in English, Chinese and Cantonese, code, math, JSON), 400 samples of 16 tokens each at temperature 0.6, top-p 0.95, with the MTP head and the drafter vs without drafting, plus a second run without drafting on other seeds as the noise baseline. The summed total-variation distance of the per-position token distributions is 4.77 with MTP against 5.02 between the two runs without (chi-square per cell 0.87 vs 0.83), and in server mode with 4 concurrent requests 4.33 vs 4.37. Drafting stays inside the run-to-run noise.

Files

• model.safetensors — the v1.1 head, bf16 (mtp.* tensors); sha256 in SHA256SUMS. The v1.0 head is under the tag v1.0.
• config.json — shapes, the target model and the sha256 of the v1 weights the head was trained against, what the head reads from the target (the last hidden state after the final norm, and the embedding of the next token), and the draft length.
• LICENSE, NOTICE, LICENSE-Qwen3.8, SHA256SUMS.