Qwen3.8 Flash Next — MLX 8-bit with native MTP

A uniform 8-bit MLX conversion of Qwen/Qwen3.8-Flash-Next, rebuilt directly from the official BF16 checkpoint with the model's native MTP draft block preserved.

Original model · Qwen overview · MLX-VLM · Qwen Community License 1.0

About this conversion

This repository contains the standard uniform 8-bit release. Eligible language-model and native-MTP modules use MLX affine quantisation at group size 32. Multimodal modules and MoE router gates remain in BF16. Group size 32 also covers the model's 160-wide hashed n-gram embedding tables.

The upstream tokenizer, chat template, vision processor, generation configuration, licence, and native MTP configuration are retained.

[!IMPORTANT] Use a runtime with explicit qwen4_exp and native-MTP support. A runtime that does not construct the Qwen4Exp MTP module may reject the 76 MTP tensor entries during strict loading.

Download and use

python -m pip install --upgrade huggingface_hub


hf download TensorFold/Qwen3.8-Flash-Next-MLX-8bit-MTP \
  --local-dir ./Qwen3.8-Flash-Next-MLX-8bit-MTP

In a compatible oMLX build:

1. Add the downloaded directory to the oMLX model directories.
2. Refresh the model registry and select Qwen3.8-Flash-Next-MLX-8bit-MTP.
3. Enable native MTP and set the draft depth to three tokens.
4. Load the model and start a chat or use the OpenAI-compatible API.

The model can run with native MTP disabled for baseline or troubleshooting comparisons. MTP changes the decode path, not the model's intended greedy output.

Apple M3 Studio performance

Benchmark environment: oMLX 0.6.3rc3 (build 2475) on an Apple M3 Studio. Measurements use greedy decoding, seed 6330, identical prompts, a separate warm-up, and 3 measured 512-token runs per mode.

The measured throughput delta with MTP enabled was 24.05%. Exact-output parity: Passed. All sustained runs reached 512 generated tokens: Passed. Coherence checks: Passed. MTP telemetry capture: Passed.

Draft acceptance is reported beside throughput because a high acceptance rate does not by itself prove a speed improvement. Results vary with prompt length, context growth, cache state, runtime version, memory pressure, and thermal conditions. The first request after loading includes model and kernel warm-up and is excluded from the steady-state result.

Runtime compatibility

Older builds without explicit qwen4_exp native-MTP support are not compatible with this checkpoint. Vision input support also depends on the selected MLX-VLM and oMLX versions; the performance figures above cover text generation only.

Architecture

Qwen3.8 Flash Next is a causal language model with a vision encoder. Its language stack combines Gated DeltaNet, Qwen Sparse Attention, sparse mixture-of-experts layers, widened gated residual streams, hashed bigram and trigram embeddings, and a native next-token-prediction block for speculative decoding.

For upstream evaluations, intended use, limitations, safety guidance, and the full architecture discussion, see the original model card.

Conversion and validation

• The converter read the official BF16 checkpoint directly.
• Eligible modules were quantised uniformly at 8 bits using affine quantisation and group size 32.
• Multimodal modules and MoE router gates were retained in BF16.
• Structural validation of all 3,747 indexed tensors, all 42 weight shards, and all 76 native-MTP tensor entries: Passed.
• Exact deterministic MTP-off/on parity: Passed.
• 3 sustained 512-token runs per mode: Passed.
• MTP drafted/accepted telemetry: Passed.
• Coherent long-generation checks: Passed.

This is a community conversion, not an official Qwen release.

Limitations

• Quantisation can reduce quality relative to BF16. The effect is usually strongest at 2-bit precision.
• The native 262,144-token context does not guarantee that every Apple-silicon system can prefill or decode that length within available unified memory.
• Native MTP needs a compatible runtime and may not improve every prompt or context length.
• The checkpoint is an MLX release for Apple silicon. It is not a GGUF, CUDA, TensorRT-LLM, or vLLM checkpoint.
• Upstream model limitations and safety considerations still apply.

Licence and attribution

The upstream model is released under the Qwen Community License 1.0. The required licence text is included in this repository and should be reviewed before use or redistribution.

Model design, training, evaluations, and upstream documentation belong to Qwen and the original contributors. The MLX conversion, native-MTP preservation, Apple-silicon validation, and packaging are provided by TensorFold.

Choose for your Mac

64GB Macs · 128GB Macs · 256GB Macs

No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.

Runtime and evidence

oMLX version mentioned in the existing card: 0.6.3rc3; consult its compatibility notes for whether this was tested and any required integrations. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.

Quick start and demo prompt

hf download TensorFold/Qwen3.8-Flash-Next-MLX-8bit-MTP --local-dir ./models/Qwen3.8-Flash-Next-MLX-8bit-MTP

Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.

Try this in a new chat with a 128-token output limit:

Explain why the sky looks blue in three short sentences.

This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.

Follow TensorFold for new Apple Silicon releases and fixes.