Beam-1.0-501B-A23B-MXFP4: a 501B model on 2× AMD MI300X

An MXFP4 (OCP Microscaling FP4) quantization of reflection/Beam-1.0-501B-A23B, built for AMD Instinct GPUs.

Beam's official BF16 (1.0 TB), FP8 (506 GB) and NVFP4 releases target NVIDIA hardware. The FP8 checkpoint alone is larger than the 384 GB of HBM on two MI300X. This checkpoint is 278 GB, so the full 501B-parameter model serves on two MI300X (TP2) with room for a 1.08M-token KV cache, enough for Beam's full 1M-token context.

New: Beam on one MI300X. We also pruned 40% of the routed experts (152 of 256 kept per layer, chosen on Beam's own reasoning), giving a 172 GB checkpoint that serves on a single MI300X: Pruned40-REAP and Pruned40-Freq. Results in their model cards and in the Space.

Results

Served with vLLM on 2× AMD Instinct MI300X (TP2), using AITER's Triton W4A16 MoE kernel.

Accuracy (Reflection's eval settings: reasoning_effort=max, T=0.7, top_p=0.99, 32k token budget)

All 5 AIME misses are completions that hit the 32k token budget; every AIME answer that finished was correct. Truncations are scored as wrong. The full transcripts are in zouaynib/Beam-1.0-501B-A23B-MXFP4-eval-traces. Reference numbers are from the Beam model card; our sample counts are smaller, so differences of 1–2 points are within noise.

More room to think. The answers cut off at 32k (5 on AIME, 10 on GPQA) were sampled again with a 64k budget (same seeds; GPU batching makes them fresh samples), everything else kept: AIME 95.8% → 96.7% (2 still ran out at 64k), GPQA 89.4% → 90.4% (BF16 reported: 90.6%).

Long context: needle in a haystack up to ~1M tokens (2× MI300X)

A passphrase hidden at 10%, 50% and 90% depth in Wikipedia text (WikiText-103), reasoning_effort=low, 2,048-token answer budget. 10/12 found. Script: scripts/needle_test.py.

Distance from BF16

Measured during quantization by running BF16 and MXFP4 copies of the hidden states side by side through all 52 layers, over 65,504 tokens (2,048-token blocks):

Expert weight SQNR is 19.0 dB in every one of the 49 MoE layers (gate/up 19.03 dB mean, down 19.00 dB, worst 18.94 dB). Per-layer metrics and per-expert routing counts are in quantization_metrics.json.

Serving throughput on 2× MI300X (1k-token prompts, 1k-token outputs)

Long context (1k-token outputs): 32k-token prompts × 4 concurrent: 19.2 ms per output token; 120k-token prompt: 14.8 ms per output token, 25.5 s to first token.

Use --attention-backend TRITON_ATTN. vLLM's default ROCm attention backend decodes Beam's long contexts much more slowly on MI300X:

What is quantized

• Routed experts (49 MoE layers × 256 experts × gate/up/down, 98.3% of all parameters) → MXFP4: FP4 E2M1 values with one shared E8M0 scale per 32 weights (4.25 bits/weight).
• Kept in BF16: attention (q/k/v/o, output gate), the 3 dense MLP layers, the router (FP32), norms, embeddings and lm_head. This mirrors the official NVFP4 recipe, which also leaves attention and the router unquantized.
• Scale selection: each 32-weight block picks between two shared exponents (the smallest one that avoids clipping, and the next one down, which clips the largest values but gives finer resolution), whichever has the lower squared error. This gains about 0.5 dB SQNR over plain OCP rounding at no cost to the format. No calibration data is used, and the format is plain OCP MXFP4, so any MXFP4 kernel can consume it.
• Layout: per-expert tensors ...experts.{e}.{gate,up,down}_proj.weight (uint8, two FP4 values per byte, low nibble first) and ...weight_scale (uint8 E8M0), with "quant_method": "mxfp4" in config.json. vLLM's generic MXFP4 MoE loader reads it directly.

Serve on 2× MI300X with vLLM

Requires vLLM ≥ 0.33 for ROCm plus three small ROCm fixes (patch, being upstreamed). Without them, Beam's Triton kernels pass CUDA-only launch options that ROCm's Triton rejects, and MXFP4 MoE on gfx942 falls back to slow emulation.

pip install --pre vllm --extra-index-url https://wheels.vllm.ai/rocm/nightly/rocm100/
# apply rocm/0001-*.patch to the installed vllm package (see rocm/README.md)

VLLM_ROCM_USE_AITER=1 vllm serve zouaynib/Beam-1.0-501B-A23B-MXFP4 \
  --served-model-name Beam-1.0-501B-A23B-MXFP4 \
  --tensor-parallel-size 2 --attention-backend TRITON_ATTN --max-model-len 131072 \
  --reasoning-parser beam --tool-call-parser beam --enable-auto-tool-choice

Use Beam's recommended sampling settings: reasoning_effort="max", temperature=0.7, top_p=0.99.

Reproduce

Everything that produced this checkpoint and its numbers is in scripts/:

• quantize_beam.py: streams the 1 TB BF16 checkpoint shard by shard (it never needs to fit on disk), quantizes the experts, and at the same time runs BF16 and MXFP4 copies of the hidden states through the model layer by layer to measure KL divergence.
• mxfp4.py: the quantizer (with unit-tested round-trip, round-half-to-even and nibble order).
• make_tiny.py + check_tiny_vllm.py: check the checkpoint layout against vLLM's real MXFP4 kernels on a tiny random Beam model.
• serve_beam.sh, eval_beam.py, bench_beam.sh: serving, accuracy and throughput.

Limitations

• On MI300X/MI325X (gfx942) there is no FP4 compute: the kernels unpack the FP4 weights to BF16 on the fly (W4A16), so the savings are in memory and bandwidth, not in FLOPs. On MI355X (gfx950, 288 GB of HBM per GPU), the same checkpoint can use native FP4 MoE kernels.
• Evaluations here are smaller than Reflection's (see sample counts above); treat differences of about 1–2 points as noise.
• Beam's license and usage terms apply unchanged (Apache 2.0).

Links

• Interactive write-up: zouaynib/beam-mxfp4-mi300x (Space)
• Eval transcripts: zouaynib/Beam-1.0-501B-A23B-MXFP4-eval-traces
• Base model: reflection/Beam-1.0-501B-A23B