Agens Volundr 32B v1 — Engine 16 GB

Main model card, benchmarks and discussion: Blockway/Agens-Volundr-32B-v1

A mixed-precision build of Agens Volundr 32B v1 for a 16 GB GPU, with the full 262K-token context inside a 16 GB memory budget.

This checkpoint runs only on Agens Engine. Its weight format is the engine's own and other inference stacks cannot load it. For other stacks, use the bf16 model or the other v1 builds linked from the main model card.

• About 2.85 bits per weight, mixed 2/3/4-bit. Each linear layer is stored at 2, 3 or 4 bits (group size 64 or 128), chosen per layer; the engine measures 2.82 bits per weight including scales over the 27.4B quantized parameters (9.67 GB). The output head is held in INT8.
• 262K context within a 16 GB budget. The KV cache is tiered to host memory, so the full context fits without filling the GPU. Host RAM: 32 GB or more.
• Speculative decoding with the DFlash2 drafter (the engine quantizes it at load). Greedy outputs are identical with and without the drafter.

Quickstart

docker run --gpus device=0 --ipc=host -p 30000:30000 \
  -v /path/to/Agens-Volundr-32B-v1-Engine-16GB:/model:ro \
  -v /path/to/Agens-Volundr-32B-v1-DFlash2:/drafter:ro \
  ghcr.io/blockwayz/agens-engine:1.3.5 --profile 16gb --model /model --drafter /drafter
# (:latest tracks the newest release)

# OpenAI-compatible API on port 30000
curl http://localhost:30000/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model": "agens", "messages": [{"role": "user", "content": "Hello!"}]}'

Requirements: a 16 GB GPU with a CUDA 12.9-capable driver (prebuilt kernels: compute capability 8.9 and 12.0), host RAM of 32 GB or more, Docker with the NVIDIA container toolkit. See the Agens Engine README for profiles and options.

Quality

Wider evaluation of this build (Engine 1.1.0, our bench26 harness, same settings as the v1 card; v1 bf16 in brackets): AIME 2025 avg@4 80.0 best of 4 runs / 76.7 mean [80.0 best of 2 / 79.2 mean]; MATH-500 96.4 [96.8]; IFEval prompt-strict 86.5 [87.2]; BFCL v4 parallel 94.0 [94.0]; BFCL v4 irrelevance 81.7 [82.1] (provisional: 8 of 240 answers hit a 32K client cap). The low-bit build gives up a few points on reasoning-heavy sets and almost nothing on instruction following and tool calling. Known limitation (this low-bit build): with thinking on and a long document in the prompt (32K+ tokens), this build can loop or give wrong answers (110K-token test, 10 samples: 6 correct, 3 looped; the v1 INT4 build in the same engine: 10 correct, 0 loops). For long documents, turn thinking off or use the v1 4-bit build. Agens Engine 1.1.1 adds a thinking repetition guard and a default output cap that limit the damage but do not fix this; use Engine 1.1.1 or later (ghcr.io/blockwayz/agens-engine:1.3.5). A revised 16 GB build is planned.

Decode speed with --mtp (Engine 1.3.0, MTP head v1.0) — same 16 GB profile, images off, one run per cell; accepted tokens per verify step in brackets:

Context with --mtp: 237,572 tokens on the 16 GB profile, 217,092 with image input on (peak 16.75–16.76 GB inside the cap on Engine 1.3.0 with head v1.0; 16.82 GB on Engine 1.3.5 with head v1.1, same contexts); the 24 GB profile keeps 258,052 with images and MTP. Sampled cells and classes with few prompts carry run-to-run noise.

MTP head v1.1 (2026-10-23). The head repo's current version (Blockway/Agens-Volundr-32B-v1-MTP) is self-distilled on the model's own samples; v1.0, the head the tables above were measured with, stays under its tag v1.0. In a paired re-measurement on Engine 1.3.5 with the same profiles and prompts, v1.1 commits more tokens per verify step than v1.0 (greedy; 16 GB profile: code 3.47 → 3.67, tool calls 3.06 → 3.27, agent 4.26 → 4.42, chat 2.67 → 2.71; 24 GB profile: agent 3.99 → 4.45, tool calls 3.38 → 3.63), up to about 11% faster decode. The tables above keep the v1.0 numbers; the head card has v1.1's.

GPQA Diamond: best of 4 runs, 6K-token thinking budget with forced close, temperature 0.6. HumanEval: pass@1, greedy, 1 run.

Against the bf16 model, teacher-forced on held-out text: mean KL 0.1165 nats, top-1 token agreement 90.39 %, perplexity 1.482 (bf16: 1.323).

Speed

Measured on a 48 GB GPU held to a 16 GB budget by the engine's hard memory cap (--profile 16gb), one conversation, greedy decoding, speculative decoding with the DFlash2 drafter. Tokens per second are decode throughput; "accepted" is the mean number of tokens committed per verify step.

The 219,070-token document took 151.4 s to prefill (time to first token); the conversation then grew to 239K tokens of context. Device memory peaked at 15.5 GiB (16.64 GB), inside the engine's hard 16 GiB cap. On a 16 GB GPU that also drives a display, less memory is free; use a shorter context there.

Image input is supported from Engine 1.1 (--images on; about 225K context on the 16 GB profile, 217,092 with --mtp on Engine 1.3).

The download is 25 GB because the files store the weights in 4-bit containers and keep the embeddings and memory tables at full precision. At load, the engine repacks the linear layers to their 2-, 3- or 4-bit widths (2.82 bits per weight on average) and keeps the embedding and memory tables in host RAM, so the GPU footprint stays within 16 GiB.

Files

Engine-format safetensors shards (model-0000*.safetensors, checksums in SHA256SUMS), config.json, quantization_audit.json (the bit width of every linear layer), tokenizer and chat template.

Licence

Apache-2.0. The licence files and NOTICE ship with the weights.