Beam-1.0-501B-A23B VQ GGUF

Reflection's 501B Beam with its full 1,048,576-token context, sized for one 128 GB DGX Spark. The 1.0 TB BF16 checkpoint becomes about 99.9 GB of GGUF parts: 98.3 GB that stay resident and a 1.56 GB BF16 token table that is read in rows from SSD. The routed experts use YoungAi's 8-dimensional vector quantization (12-bit, and 13-bit on the eight bottom MoE layers); everything the model needs to read on every token stays at Q4_K or source precision.

Status, 2026-10-20: the conversion is finished, verified and uploaded. Against the BF16 original, the release's own bytes agree on the next token 82.9% of the time (mean KL 0.295) over 5,491 held-out tokens, in a reference forward pass (below). The target runtime is ds4-dfm-rs; its Beam family (gated GQA + SWA attention, Harmony output parsing, FP8 KV for the full layers, SSD embedding rows) is being added, so no DGX Spark serving speed exists for this artifact yet. Memory figures are computed from the exact byte layouts plus labelled runtime estimates.

At a glance

Why this size: the 1M context decides it

DeepSeek-V4.1-Flash VQ keeps a ~106 GiB base resident and still serves 1M tokens because its attention is compressed: YoungAi measures the whole 1M KV cache at 0.89 GB. Beam's nine full-attention layers are ordinary GQA (8 KV heads × 128). At 1M tokens they need 36.0 GiB of KV in BF16, or 18.3 GiB in FP8 — twenty to forty times DeepSeek's. A Beam that copied DeepSeek's weight size would not serve 1M on one Spark; the weights have to give up what the KV cache takes.

Budget for one bank at the full 1,048,576 tokens, against the 118.3 GiB a GB10 reports available to ds4-server --check-config (the GLM-5.3 VQ quote):

That leaves about 1.7 GiB of spare quote at the full context, the same order as GLM-5.3-Flash VQ's qualified 1M point (116.2 of 118.3), and about 5.6 GiB physically free with the floor. Without the floor, Beam at 1M holds 112.7 GiB — DeepSeek V4.1 VQ's level. With BF16 KV the same table needs 134.4 GiB, so FP8 KV on the nine full layers is part of this release's contract, not an option.

The eight 13-bit layers are where the spare bytes went: each costs 0.146 GiB. All-12-bit would leave 2.8 GiB of spare quote; fourteen 13-bit layers (the DeepSeek count) would leave 0.8 GiB, Ling-3.1's tight corner. memory-budget.json and tools/memory_budget.py reproduce every row.

Recipe

• Routed experts. Every 8 consecutive weights of an expert row become one index into a codebook, times one F16 row gain (the row RMS, computed in FP64 and rounded before normalizing). One codebook per layer serves all 256 experts and gate/up/down; YoungAi measured a shared book to cost nothing against one per expert. Codewords are trained in FP32 (equidistant init, 32 Lloyd iterations with FP64 sums) on a stratified pool of 9.8M vectors per layer (20 gate, 20 up and 40 down rows from every expert), rounded FP32 → F16 → E4M3, and every vector is then assigned against the stored codebook. No text is involved.
• Which layers get 13 bits. Each extra index bit divides the weight error by about 1.19 (YoungAi: 0.201 / 0.171 / 0.143 at 11 / 12 / 13 bits), the curve is convex, so spare bytes go to many layers one bit at a time, and to the bottom first: YoungAi measured shallow layers worth 1.2–1.4× more per GB, and the GLM-5.3 end-to-end sweep ranked layers 3–16 first. Beam's eight bottom MoE layers (3–10) take the bytes the 1M budget leaves.
• Everything read on every token besides the experts — attention q/k/v/o and the output-gate projection of all 52 layers, the three dense FFN layers and the output head — is Q4_K, as in DeepSeek V4.1 VQ. YoungAi measured that skeleton's error at 0.5% against 17% for the experts: raising it buys little.
• Not touched: router weights and correction biases, all norms, the per-head attention gate bias, and the token embedding (BF16, read from SSD).

Weight-space reconstruction of the routed experts, measured by the converter on the stored bytes against the BF16 source (relative MSE; not a model-quality measurement): the eight 13-bit layers 0.1441–0.1446, the 41 12-bit layers 0.1698–0.1703 — cosine ≈ 0.925 / 0.911, the same as DeepSeek V4.1 VQ's 0.925 / 0.911. Held-out rows give the same values. Every layer is in weight-error-by-layer.json; the per-matrix records are in conversion-receipts/. The Q4_K tensors average 0.55% relative MSE (worst 0.67%, layers.0.mlp.down_proj; output head 0.53%).

Measured on the converted bytes

A teacher-forced comparison with the BF16 original, run layer by layer on an H100: the release side reads every weight from this repository's files (VQ experts restored from the stored indices, Q4_K dequantized, BF16 embedding rows); the original side reads the source checkpoint. Both run BF16 attention, so FP8 KV is not part of these numbers, and this is not the ds4 engine. The forward pass re-implements transformers 5.20.2's Beam layer and matches BeamDecoderLayer (layer 5 bit-identical, layers 0 and 3 within 1.1e-4 relative RMS).

For scale, on different texts: the same method's bare DeepSeek V4.1 Flash base agreed on 69–79% of next tokens (KL 0.42–0.69, YoungAi's rulers); GLM-5.3-Flash VQ reached 81.4% (KL 0.288) only after its 19 most sensitive layers got an 8-bit residual stage. The exact top-8 expert set matches the original's for 54% of tokens at layer 3 and 3–5% in the last layers — routing is hash-like, as YoungAi found on DeepSeek — yet the outputs still agree.

Kernel budget on GB10 (99 KiB shared memory per block)

GB10 allows 101,376 B per block. Beam's down projection reads 8 experts × 2,560 BF16 mid values (40 KiB), so the 13-bit layers need the down/verify kernels to stage the experts in two groups of four (one extra barrier per block); the 12-bit layers fit the GLM/Ling kernel scheme as it is, with 27 KiB to spare, enough for a later 8- to 11-bit residual stage in decode. The figures are the ds4-dfm-rs GLM/Ling kernel formulas applied to Beam's shapes; those kernels still need their row widths generalized to 5,120 and 2,560.

What stays native

• Context: 1,048,576 tokens. Sliding layers use RoPE (θ = 10,000) over a 512-token window; full layers use no positional encoding and scale the query by (1 + (pos + 1) / 2048) ** 0.1. Nothing is rescaled or extended.
• Chat template: the original chat_template.jinja (Harmony wire format with XML tool-call bodies, analysis and final channels, tools declared as TypeScript, <|constrain|>xml calls with rfl:parameter bodies, tool results on analysis). reasoning_effort accepts minimal / low / medium / high / xhigh / max = 8 / 32 / 64 / 128 / 256 / 512 and defaults to 512. Generation stops on <|return|> (151674) or <|call|> (151675).
• Sampling: Reflection evaluated with temperature 0.7 and top_p 0.99 (generation_config.json).
• Numerics kept in FP32 by the source model: router logits, SwiGLU, logit soft-capping at 30.

Files

These are independent GGUF parts, not numbered GGUF splits: do not concatenate them. Standard llama.cpp, Ollama and Transformers loaders do not read this format.

Upstream scores

Reflection's scores for the original checkpoints, from the Beam model card. They describe the source model, not this quantization.

Limits

• This is lossy: about 1.5 bits per routed-expert weight. The release agrees with the original's next token 82.9% of the time on the held-out set above; Korean prose is its weakest case (75.1%, perplexity 2.28 → 3.65). GLM-5.3-Flash needed more bits on its bottom layers to stop collapsing in Korean; behaviour in long free generation has not been checked here.
• Serving needs the ds4-dfm-rs Beam family, which is not merged yet. Until it is, these files are for inspection and runtime development.
• The 1M budget assumes FP8 KV on the full-attention layers and 2,048-row prefill chunks at long context; scratch is an estimate until measured on GB10.
• Uniform zero-corpus quantization: no domain sidecar, no calibration.

Credits and sources

• Reflection — Beam-1.0-501B-A23B, Apache-2.0.
• YoungAi by Wenzhou Wu (吴文周, fodelf / wenzhouwu), MIT — the method: 8-dim VQ with per-layer shared E4M3 codebooks, F16 row gains, the 12 + 1 bit-plane layout for 13-bit layers, shallow-layers-first bit allocation and the zero-corpus rule (docs/TECHNICAL.md §4.1, docs/113.md). The converter's CUDA assignment and FP64 Lloyd follow YoungAi gguf-tools/quantize/v41_vq.cu; its notice is kept in LICENSE-YoungAi.
• ds4-dfm-rs — the GVQ1 container, its GB10 kernels, and the Ling-3.1 / GLM-5.3 VQ tools this converter is forked from.
• llama.cpp / GGML — the GGUF container and the Q4_K layout.

License

Apache-2.0, as the source model. The quantization tools derive from YoungAi and ds4.c (MIT, LICENSE-YoungAi, which includes the GGML notice).