Kimi K3 — Merged 2-Expert Model (66.3 GB)

Fully dense, 62.23 B-parameter model produced by collapsing all 896 routed

experts of each MoE layer of Kimi K3 into 2 super-experts using the

MergeMoE expert-output-merging technique.

Derived from: Moonshot AI's Kimi K3

(open weights, Apache 2.0) via Unsloth's 1-bit GGUF conversion

(UD-IQ1_S, 14 shards).

Paper: Kimi K3 Under Compute Constraints: Implementing MergeMoE to Take 14 Shards to 5GB on Consumer Hardware

Author: Haitham al-Waisy · Contact: haithamalwaisy@gmail.com · X/Twitter · YouTube

Code: https://github.com/Haitham-alwaisy/Kimi-K3-MergeMoE

Model anatomy

• 93 blocks, all 896 experts per layer → 2 super-experts (usage-frequency-weighted average)
• All-Q8_0/F32 weights (1,392 Q8_0 + 1,181 F32 tensors)
• 62.23 B parameters, 100% active per token

Intended use

This model is intended as a base for fine-tuning and Reinforcement Learning.

It is recommended to be run inside the Prime Agent harness.

Usage

hf download HaithamalWaisy/kimi-k3-merged kimi-k3-merged-2exp.gguf --local-dir .

License

Apache 2.0 (inherited from Kimi K3). Quality was not benchmarked by this project.