Qwen3.8-Flash-Next for QEngine

Qwen3.8-Flash-Next in the weight layout of QEngine, a source-available inference engine for 3× NVIDIA RTX 3090. Download it and run it with QEngine directly; there is no conversion step.

This revision is for QEngine 1.3.4.1 or later. It is about twice as close to the original BF16 model as the 1.3.4 files: the parts every token passes through are more precise, and the routed experts are unchanged. The previous files remain available as revision v1.3.4 (see Previous version). These files only work with QEngine; QEngine 1.3.4 and older stop at load on them with tensor not found: ...linear_attn.in_proj_qkv.weight_packed (update QEngine, or download --revision v1.3.4). The quantization_config in config.json only describes the files; QEngine reads the formats from the tensors.

Closeness to BF16

We measured next-token KL divergence against the official BF16 model on a public reference set: 21 cases and 32,359 positions covering prose, code, Japanese, Chinese, SQL, agent turns and WikiText-2. The set was scored with its own compare.py (dataset revision 5294aa9a); every position runs through QEngine's decode path in forwards of up to 16 tokens on 3 cards. Lower KL is closer to BF16.

With the context built by the server's batched prefill (the last 512 positions of the long cases), mean KL is 0.146, against 0.358 for v1.3.4.

In the club-3090 8-pack (150 scenarios, 3 runs, 3× RTX 3090) QEngine 1.3.4.1 with these files passed 117.0 (thinking off) and 124.7 (thinking on) on average; with the 1.3.4 files it passed 114.7 and 128.7. The run ranges overlap, and three runs cannot separate the two versions.

Speed. The more precise DeltaNet projections are read on every decode step, so on the same engine version decode is about 4-5 % slower than with v1.3.4 (prefill is unchanged). QEngine 1.3.4.2 wins this back: on 3× RTX 3090 with these files it decodes 314 / 401 tok/s single-stream (club-3090 narrative / code; 1.3.4.1: 292 / 368) and 1,022 tok/s with 8 concurrent agents (1.3.4.1: 897), with the same quality. Use QEngine 1.3.4.2 or later.

What's inside

• Converted from Intel/Qwen3.8-Flash-Next-W4A16-AutoRound, an INT4 quantization of Qwen/Qwen3.8-Flash-Next.
• base.safetensors, layer-00 … layer-47: the model, regrouped per layer for QEngine:
  • routed experts: INT4 from the AutoRound checkpoint, repacked bit-exact;
  • DeltaNet projections (in_proj_qkv / z / out_proj): stored as INT8 g128 (int8 weights + bf16 scales per 128 inputs, symmetric), exactly what the engine runs (v1.3.4: INT4); the download is about 1 GB larger;
  • shared expert and n-gram key/value projections: INT4 (group 128), calibrated with GPTQ on our own calibration text (v1.3.4: round-to-nearest);
  • QSA attention q/k/v/o and lm_head: BF16 in the files, INT8 in the engine;
  • hyper-connections, router, embeddings, indexer, gates, norms: BF16.
• mtp.safetensors / mtp-int4.safetensors: the multi-token-prediction head used for speculative decoding (QEngine runs its attention / fc from the BF16 copies as INT8, its experts and shared expert INT4).
• ngram_int8.safetensors: the n-gram embedding table in INT8 (51.8 GB instead of 102.4 GB in BF16). In QEngine's tests its next-token KL against the BF16 table was within the engine's own run-to-run noise.
• expert_prior.bin, cold_experts.json, token_rank.bin: routing statistics and the draft vocabulary that QEngine uses.

Requirements

• 3× RTX 3090 (24 GB), or 2 of them. GPU-to-GPU P2P helps but is not needed (see the QEngine README).
• Host RAM: with about 140 GB (3 cards) or 160 GB (2 cards) QEngine uses its full defaults. Smaller hosts (from 64 GB) get a plan sized to their RAM at start-up. Details are in the QEngine README.
• About 129 GB (120 GiB) of disk (v1.3.4: 128 GB).

Previous version

The 1.3.4 files (smaller DeltaNet projections, about 4-5 % faster decode on the same engine version, further from BF16) are kept as revision v1.3.4:

hf download bpx03/Qwen3.8-Flash-Next-QEngine --revision v1.3.4 --local-dir models/Qwen3.8-Flash-Next-QE-INT4-v1.3.4
server/run.sh 8080 --model models/Qwen3.8-Flash-Next-QE-INT4-v1.3.4

QEngine 1.3.4.1 and later run both revisions: the engine reads the format from the files, so nothing else changes.

License

The weights are a derivative of Qwen3.8-Flash-Next and are distributed under the Qwen Community License 1.0 (copy included). QEngine itself has its own license, see its repository.