Which version should I use?

Numbers are copied from each repository's own model card; the references differ between rows, so agreement figures are not directly comparable. "Not tested" means no such measurement exists. Sizes marked "repo files" are the total size of files in the repository.

How to choose

• Lowest RAM: ternary-mlx-K50. Measured peak footprint about 22.5 GB with the resident expert cache on (15.0 GB with the cache off, but much slower). Top-1 agreement 91.9%, but from only 8 in-sample sequences and against a different reference (the unpruned ternary model). Needs its dedicated Python loader.
• Out of the box, most thoroughly measured: min3bit-oMLX. Runs directly in oMLX. MLX peak about 34.8 GB, top-1 agreement 0.840 on 32 prompts (reference: the unpruned GSQ-RCO Q2 build, not the FP8 model), median about 8.0 tok/s decode.
• mixed-mlx-K50: output is equivalent to ternary-mlx-K50 (256/256 generated tokens identical). Needs the C++ runner in Jakevin/moe-offload. Process RSS peak about 23.3 GB at a 15.6 GB expert budget; top-1 agreement against a reference has not been measured.

Agreement was measured differently for each version, so the numbers cannot be compared directly. The ternary versions have not yet been scored on the same 32-prompt set as min3bit-oMLX.

Qwen3.8-Flash-Next-mixed-mlx-K50 (mixed-precision, expert-pruned)

Runtime: the 2t expert format is read by the open-source C++ runner in Jakevin/moe-offload (mlx-qwen38/cpp). Standard mlx-lm / mlx-vlm cannot load this repository directly, and the Python runtime.py / runtime_standalone.py shipped with the ternary K50 repository cannot read 2t records. See "How to run" below.

This is a mixed-precision re-packaging of Jakevin/Qwen3.8-Flash-Next-ternary-mlx-K50. It contains no ternary (base-3 packed) tensors: every routed expert is an MLX affine 2-, 3- or 4-bit tensor.

What changed versus the ternary K50 repository

• Experts that were stored as ternary (5 trits per byte, one bf16 scale per 64 weights) are converted losslessly to a 2-bit MLX affine record, kind 2t: codes in {0,1,2} in MLX uint32 2-bit lanes, group_size=64, bf16 scale equal to the ternary scale, bf16 bias equal to -scale. Each weight scale*code - scale equals the ternary value scale*{-1,0,+1} exactly.
• All other experts (2/3/4-bit affine), attention, shared experts, embeddings, lm_head, routers and the PLE table are byte-for-byte the same as the ternary K50 repository.
• Expert pruning is unchanged: layer 0 keeps all 512 routed experts, layers 1–47 keep 256 of 512 (50% of routed experts removed). Kept original ids are in K50_kept.json.

Equivalence with the ternary K50 model

Measured with the C++ runtime, greedy decoding, one Python coding prompt and one English prose prompt, 256 generated tokens each:

• Generated tokens: 256/256 identical to the ternary K50 model on both prompts.
• Logits: maximum absolute difference 0 over the first 16 decode steps.

Speed and memory (C++ runtime)

Machine: Mac mini, Apple M4 Pro, 64 GB. Single process, MLX cache limit 2 GiB, greedy decoding, 256 generated tokens per prompt, resident expert budget 15.6 GB (hot experts stacked in memory plus an LRU for the rest, remaining experts read from SSD), one synchronisation per layer.

• Decode-time expert cache hit rate: 95.4% (coding), 96.4% (prose), same as the ternary K50 model.
• In the interleaved runs the speed difference between the two formats is within run-to-run variation; the formats should be treated as roughly equal in speed.
• With every expert resident (about 23.1 GB of experts, no SSD reads) the same runtime measured 1.71 tok/s (coding) and 1.86 tok/s (prose), slower than the 15.6 GB budget; the cause has not been investigated. Process RSS in that mode was about 30.6 GB.
• Memory at the 15.6 GB budget: process RSS peak about 23.3 GB, MLX peak 19.6 GiB, no swap growth.

How to run

Requires Apple Silicon, CMake ≥ 3.24 and an MLX C++ build (see the runner README for MLX_ROOT).

huggingface-cli download Jakevin/Qwen3.8-Flash-Next-mixed-mlx-K50 --local-dir ~/models/k50-2t

git clone https://github.com/Jakevin/moe-offload.git
cd moe-offload/mlx-qwen38/cpp
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release   # add -DMLX_ROOT=/path/to/mlx if needed
cmake --build build -j --target qwen38_decode

# header-only check of every tensor
./build/qwen38_decode --model ~/models/k50-2t --check-weights

# greedy decode; prompts.json = {"prompts": [{"ids": [...]}]} (token ids from this model's tokenizer)
./build/qwen38_decode --model ~/models/k50-2t --ids prompts.json --prompt-index 0 --n 256 \
    --resident-gb 15.6 --stack-mode hot --out out/run.json

--resident-gb sets the memory for resident experts (hot stack + LRU); lower it on machines with less than 64 GB. Full details: runner README.

Files

Limitations

• Apple Silicon and MLX only.
• Needs a runtime that understands the 2t record kind, such as the C++ runner in Jakevin/moe-offload.
• Inherits all limitations and evaluation caveats of the ternary K50 model; see its model card.
• No benchmark has been run on this model. Speed numbers are from one machine and two prompts.

License

Weights are under the Qwen Community License 1.0. See LICENSE (the original license file, unchanged) and NOTICE.

This model is a derivative work of Qwen/Qwen3.8-Flash-Next. Copyright (c) 2026 Qwen. As required by the license, the copyright notice and permission notice are included in LICENSE. If this model or a derivative is used in a commercial product or service with more than 100,000,000 monthly active users or US$20,000,000 monthly revenue, the model name must be prominently displayed in that product's user interface. Model-as-a-Service or AI Work Assistant businesses need a separate license from Qwen for commercial use; see LICENSE for the full terms.