Beam 1.0 501B-A23B, streaming pack, NVFP4 experts / MLX 8-bit residents

This is Reflection's model. This pack is a repack of reflection/Beam-1.0-501B-A23B: 501.3B parameters, 23.4B active, 52 layers (3 dense layers and 49 layers of 256 routed experts, 8 active for each token). The weights, the architecture and the tokenizer belong to Reflection. Reflection releases them under the Apache License 2.0. The LICENSE file here is theirs.

The pack is made from Reflection's NVFP4 checkpoint, reflection/Beam-1.0-501B-A23B-NVFP4 (revision 3bd1dccf13c8f7aa3d6732c5aaa48ba107a2c086, ModelOpt format). The draft model is Reflection's reflection/Beam-1.0-501B-A23B-DFlash2 (revision 163b276be744c8ce2b8385ea85de54edb3088211, Apache License 2.0). This pack adds no training and no fine-tuning. The routed experts keep the release's NVFP4 codes and block scales, in a streaming layout. The trunk's projections are quantized to MLX 8-bit (the q8 residents, below).

What this pack holds

• Routed experts (49 layers × 256 experts), NVFP4, in experts.bin (277.5 GB). Each (layer, expert) pair is one record of 22,118,412 B: the gate, up and down projections, each with its E2M1 codes (U32 words), its E4M3 block scales (one for each 16 inputs) and its F32 global scale. A record starts every 22,122,496 B (4 KiB alignment): 12,544 records. expert-manifest.json gives the format (mlx-stream-nvfp4-expert-bank-v1, layout nvfp4-gud-9part), the source pin, the 9 parts of a record, and the offset, length and sha256 of each record. The codes and the block scales are the release's bytes. The release's activation scales (input_scale) are not used: the activations stay BF16.
• The global scales. MLX computes the scale of an NVFP4 block as e4m3(block scale) × (g / 2688) in f32. For each projection the pack stores the f32 value g for which f32(g / 2688) equals the release's weight_scale_2. For 4,878 of the 37,632 projections no such g exists. For these the pack stores the nearest g, 1 f32 ulp (about 7e-8 relative) from the exact value. The manifest lists them (global_scale_inexact).
• The residents (the trunk and the head), in model.safetensors (9.44 GB, 1,274 tensors). Each linear of the trunk and the head is MLX affine 8-bit with group size 64 (U32 codes, BF16 scales and biases): the attention's q, k, v, gate and o projections of each layer, the gate, up and down projections of the 3 dense layers, and the head. The embedding, the norms, the attention's gate biases and the routers (F32) are as released. config.json names the tier ("residents_format": "affine8_g64"). The index gives the sha256 of each tensor.
• The q8 rule. For each group of 64 values, the bias is the group's minimum (a BF16 value of the release, so it is exact). The scale is the smallest BF16 value that is not less than (maximum - minimum) / 255. Each code is the nearest step (ties to even). Thus no value is clipped, and each value is within half a scale of the release's value.
• The resident seed, expert-usage-v1.json. For each routed layer, a count of the experts that two requests routed. The two requests are not the benchmark's: an explainer prompt (16,695 prompt tokens) and a prose prompt (16,456 prompt tokens), 2,048 generated tokens in all. At the load the server fills each routed layer's prompt rows with the layer's most used experts from this file. Thus the first request does not start cold.
• The draft model, drafter.safetensors and drafter-manifest.json. drafter.safetensors is the DFlash2 release's model.safetensors, byte for byte (1.93B BF16 parameters, 5 layers). drafter-manifest.json (format mlx-stream-dflash2-drafter-v1) gives the release pin, the drafter's config and the sha256 of each tensor. The server finds the pair by these names in the pack directory and reads it only for the draft lane (mtp_lane: true, below). It then checks the manifest against the trunk, and the sha256 of each tensor and of the file. A pack without the pair refuses mtp_lane: true by name.

The q8 residents against the release's BF16 residents, on layers 0 to 5 of Beam at their real geometry: the q8 residents move a row's logits by 1.32 times the model's own BF16 rounding (the median of the ratio), and they keep the BF16 reference's argmax on each row with a clear margin (170 of 170). Their relative RMS error on Beam's weights is 0.54 to 0.63 % (MLX's own quantization gives 0.69 to 0.84 %). A decode step reads 7.62 GB of q8 residents in place of 14.35 GB of BF16 residents.

What it is for

This pack is served by mlx-serve with the mlx-stream plugin (beam arch). The residents load one time. The routed experts that the router selects stream from experts.bin on the SSD into a bounded cache of expert rows on the GPU. Thus a Mac with 256 GB of memory serves the model, although it cannot hold the 277.5 GB of experts. The host renders the release's chat template, returns the analysis channel as reasoning_content and the final channel as content, and parses tool calls, on /v1/chat/completions and /v1/messages.

Other loaders (Transformers, vLLM, SGLang, mlx-lm) do not read this layout. Use Reflection's repositories with them.

How to run

Beam support is in two open pull requests:

1. Build the server. Put the two checkouts side by side, then in the mlx-stream checkout run zig build serve -Dmlx-serve=../mlx-serve (zig 0.17.0; the plugin's README, "Building and testing"). The host's own build requirements are in mlx-serve's README.

2. Set the GPU's wired limit (on a 256 GB Mac): sudo sysctl iogpu.wired_limit_mb=245760.

3. Download this repository to a directory on the internal SSD with hf download OpensourceWTF/Beam-1.0-501B-A23B-NVFP4-q8-mlx-stream --local-dir <dir>. The server reads experts.bin past the page cache and does not follow a symlink: the file must be a regular file or a hard link (the Hub cache's symlinks do not load).

4. Pin the rows in ~/.mlx-serve/model-settings.json. The key is the path that --model names:

   {"/path/to/Beam-1.0-501B-A23B-NVFP4-q8-mlx-stream": {"expert_prefill_rows": 197, "expert_rows": 207}}
   

5. Start the server:

   MLX_SERVE_GPU_CEILING_MB=245760 mlx-serve --model /path/to/Beam-1.0-501B-A23B-NVFP4-q8-mlx-stream --serve \
     --port 11391 --ctx-size 32768 --no-vision --no-warmup-eager --prefix-cache-entries 0 --metrics --log-file off \
     --kv-quant off --wired-margin-gib 2
   

   The beam arch refuses a quantized KV cache (BeamKvQuantUnsupported), so --kv-quant off is necessary. The release's sampling is temperature 0.7, top_p 0.99 and repetition penalty 1.0 (generation_config.json).

The rows are the expert slots that the GPU keeps for each routed layer: the prompt pass uses the prompt rows, and the decode steps use the decode rows. A row is one expert record in each of the 49 routed layers (1,084,002,304 B). One more prompt row and one more decode row together cost about 1.09 GB of process memory.

Memory on a 256 GB M5 Ultra

The server loads pinned rows as you set them. It does not compare them with the free memory. When the pin is more than its own fill, it writes forced rows ... exceed the fill by N bytes ...; proceeding (operator pin) and continues. On a machine with more other memory, remove 3 prompt rows and 3 decode rows for each 3 GB of more non-file memory. If the free memory goes near 0, the system pages and decode slows down. If you set no rows, the server's fill takes the most rows that fit the target beside the non-file memory. The fill's terms are upper bounds, so it takes fewer rows than this pin.

Context

The measurements are for one prompt of 16,445 tokens and 1,024 generated tokens, at a billed context of 32,768 tokens (--ctx-size 32768). The server reserves the KV cache for the full context at the load. The 9 full-attention layers keep 36,864 B of KV for each token (1.21 GB at 32,768 tokens). The 43 sliding-window layers keep a ring of the 512-token window. Thus each 32,768 tokens of more context take about 1.2 GB, about one prompt row and one decode row. A prompt longer than the billed context refuses (ContextOverBill). The arch keeps no prefix cache, so each request runs its full prompt. The model's own context is 1,048,576 tokens. Other prompt lengths and contexts are not measured for this card.

Measured

One Mac Studio M5 Ultra (256 GiB, internal SSD), 2026-10-18. The SSD read 13.0 GB/s in a cold prompt pass. Each run is a fresh server and its first request: a coding task (16,445 prompt tokens in Beam's chat template, 1,024 generated tokens, all in the analysis channel), the release's sampling (temperature 0.7, top_p 0.99), seeds 1234 and 4321. The load filled 197 of 197 prompt rows of each routed layer from expert-usage-v1.json. KV cache BF16, rows 197 / 207, host 0c1802e7, plugin 2769fbd. Before each load the harness removed its own files from the page cache.

Cold and seeded first requests

A fresh server does not start cold with this pack: before the first request, the load reads the seed's records (at 183 / 193 rows: 8,967 records, 198.3 GB, in 14.0 s). Without the file, the load fills each layer's prompt rows with the layer's lowest-index experts. The first request's prompt pass depends on the experts that the rows hold at the load. Measured at 183 / 193 rows, seed 1234, one run each, on plugin aa60928 (the lowest-index row: 6791939, which adds that fill):

The seed does not contain the coding task: it comes from the two other prompts. The three runs give three different texts, so their decode rates are not a comparison of the seeds.

The draft lane (opt-in)

The pack holds Reflection's DFlash2 draft model. With mtp_lane: true the server decodes in rounds. The drafter proposes 3 tokens (draft_depth 3). The trunk verifies them in one pass of at most 4 rows. The accepted tokens and the trunk's next token then commit. The drafter's memory (4.1 GB) is 4 rows: at equal memory the lane runs at 193 / 203 rows.

The lane is off by default. Its default rule, exact, is lossless: each token is the target's own draw. At equal memory (183 / 193 rows against 179 / 189) exact gave +1.9 % and +4.1 % decode (the two seeds) and -3.0 % prefill. That does not meet the keep rule (+2 % decode on both seeds and no prefill loss), so the lane stays off. The rule typical (Medusa's typical acceptance) also accepts a draft to which the target gives sufficient probability. It is lossy: it is an opt-in. The settings of the measured opt-in:

{"/path/to/Beam-1.0-501B-A23B-NVFP4-q8-mlx-stream": {"expert_prefill_rows": 193, "expert_rows": 203, "mtp_lane": true, "draft_depth": 3, "mtp_acceptance": "typical", "mtp_typical_delta": 0.6}}

At equal memory (the serial control at 197 / 207 rows, the lane at 193 / 203), with the setup of "Measured", one A B B A for each delta, the means of the two seeds:

The prefill decreases because the drafter's memory takes 4 prompt rows.

The quality over long texts: 3 prompts (the coding task, the explainer and the prose prompt; 16.4K to 16.7K prompt tokens), 2,048 greedy tokens each, 193 / 203 rows, depth 3. The serial model scored each position of each text (6,144 positions for each rule). The excess surprisal of a token is ln p(the top choice) - ln p(the token).

No rule drifts: the mean excess surprisal does not increase from the first quarter of the texts to the last (typical 0.6: 0.050, 0.047, 0.033 and 0.038 nats; 0.2: 0.135, 0.126, 0.110 and 0.107). No text degenerates: all 9 texts run to the 2,048-token cap, the host's loop guard stops none of them, and no text ends in a cycle. The exact texts differ from the serial texts only at near ties (ranks 2 to 5, none more than 1 nat below the top choice), where a verify of 4 rows and a serial step round differently.

The recommended opt-in is delta 0.6, the default of mtp_typical_delta. Delta 0.2 is 8 % faster, but its excess surprisal for each token is 2.8 times that of 0.6, 74 of its 6,144 tokens are more than 2 nats below the top choice, and its second half repeats more.

Methodology

• Each number is the first request of a fresh server. The numbers of "Measured" are one run on each seed. The draft lane's numbers are A B B A runs against the serial control (control, lane, lane, control, a fresh server for each run) on each seed. The server's own log lines give the prefill, the decode and the SSD bytes; /usr/bin/time -l gives the peak footprint.
• The plugin's documentation of the beam arch holds every measurement on this card and the method of each: docs/beam.md at 8a9a252 (pull request #20): the serve recipe, the residents, the expert route, the draft lane and its stability, the settings, the memory bill, and the parity tests against a numpy reference of the model on a fixture and on layers 0 to 5 of the real weights.
• The converter that made this pack: scripts/prepare_nvfp4_pack.py --family beam --residents q8, and its drafter step for the draft model: docs/nvfp4-pack.md at c794640 (pull request #19). expert-usage-v1.json went into the pack after the build, with its sha256 checked. pack-sha256.txt lists the 12 pack files in the converter's form; each value was read again from the files before the upload.

Files

290.8 GB in all. Keep the files in one directory.

License

Reflection releases Beam 1.0 and its DFlash2 draft model under the Apache License 2.0. LICENSE is their file. This pack adds no training. It keeps the release's NVFP4 expert bytes (the global scales as above), quantizes the trunk's projections to MLX 8-bit, and changes how the weights are stored.