The checkpoint · Quality · What it fits on · Use it · Changelog

Qwen3.8-Flash-Next-PLE-INT6-Merlin

An INT6 copy of the per-layer embedding (PLE) n-gram table of Qwen3.8-Flash-Next, for the Merlin engine's --ple int6. Merlin keeps this 51B-parameter table in host RAM. At 6 bits it takes 38.74 GiB instead of the 47.68 GiB of the FP8 table in the checkpoint, so each server needs about 9 GiB less RAM, with output close to FP8 (see Quality). This is not a full checkpoint: Merlin lays it over the Flash-Next or Swift1.5 checkpoint you already have, and never changes those files.

_{numbers: host RAM is the size of the table Merlin loads (INT6: the eight files in this repository; FP8: the table in the checkpoint). KL and agreement: each model's INT6 against its own FP8 on the same engine and prompts, Merlin 3.0.1 on 4× RTX 3090 on 2026-11-01 (Flash-Next two runs, Swift1.5 one). Each figure is also in a table below.}

The checkpoint

Files in this repository

_{41,600,199,680 bytes (38.74 GiB) of table in all. Built from the BF16 table of Qwen/Qwen3.8-Flash-Next at revision de4b8e4d43b917e7706784d8bb445c9af86a3540 by the engine's models/flash-next/make-ple-int6-pack.py --from-bf16; the build is deterministic. Merlin builds <checkpoint>-int6 beside your checkpoint from links plus a generated config and index, so the FP8 files stay as they are and --ple fp8 keeps working.}

Quality

Each model's INT6 table against its own FP8 table, measured on the Merlin engine:

_{Merlin 3.0.1 on 4× RTX 3090 on 2026-11-01 (Flash-Next: two runs; Swift1.5: one run), the same prompts for both tables. A control with every code negated gives a KL of 5.46, so the check does see a damaged table. These are our own measurements on our hardware, not an independent evaluation. Swift1.5-Huihui did not pass: its MTP draft acceptance fell 0.15–0.17 below FP8 against a 0.10 bar, so Merlin refuses --ple int6 for it.}

What it fits on

_{All three checkpoints ship the same FP8 n-gram table, byte for byte, so one pack serves every supported one. The pack changes only host RAM: the KV pool, sessions held at once and GPU profiles are the same as with FP8.}

Use it

venv/bin/merlin pull flash-next --ple int6   # fetch, verify, build the overlay
venv/bin/merlin serve flash-next ~/models/flash-next-int6 --ple int6

Use swift1.5 in place of flash-next for Swift1.5. The pull builds the overlay folder <checkpoint>-int6 beside your checkpoint (~/models/flash-next-int6 with the default ~/models), and serve takes that folder. ./setup.sh offers the same choice. merlin pull checks every file against ple-int6-pack.sha256; after a download by hand, run sha256sum -c ple-int6-pack.sha256 inside the folder. FP8 stays the default, and --ple fp8 goes back to it at any time.

[!IMPORTANT] This pack is not a model on its own. It needs Merlin 3.0.1 or later and the Flash-Next or Swift1.5 checkpoint; stock vLLM does not read it.

Good to know

• The quality figures come from one setup: 4× RTX 3090 at 220 W with P2P. The table lives in host RAM, so other GPUs are expected to behave the same, but they were not measured with INT6.
• Swift1.5-Huihui is refused for now, not broken: a recalibrated acceptance bar is planned for a later release.
• Decode and prefill speed were not measured separately with INT6; MTP draft acceptance, which drives decode speed, is in the Quality table. The 3.0.1 bench cards use the FP8 default.
• The quality checks ran before the GPU unlock was installed, with P2P only. Merlin's 3.0.1 bench numbers are measured with Zagitta's gpu-unlock plus P2P; the unlock changes speed, not output, so these results hold with or without it.

Changes

_{Card-only edits are not listed. Engine releases: the engine's CHANGELOG.}

Credits and licence

Qwen (the model and its BF16 n-gram table; Qwen Community License 1.0, see LICENSE and NOTICE; its conditions apply, including the commercial-scale and model-as-a-service terms), RadixArk (the FP8 table this pack sits beside), and the vLLM community's PLE-offload work. The INT6 pack is by halt95.