Kimi K3 — Merged 2-Expert Model (66.3 GB)
Fully dense, 62.23 B-parameter model produced by collapsing all 896 routed experts of each MoE layer of Kimi K3 into 2 super-experts using the MergeMoE expert-output-merging technique.
Derived from: Moonshot AI's Kimi K3
(open weights, Apache 2.0) via Unsloth's 1-bit GGUF conversion
(UD-IQ1_S, 14 shards).
Author: Haitham al-Waisy · Contact: haithamalwaisy@gmail.com · X/Twitter · YouTube
Code: https://github.com/Haitham-alwaisy/Kimi-K3-MergeMoE
Model anatomy
- 93 blocks, all 896 experts per layer → 2 super-experts (usage-frequency-weighted average)
- All-Q8_0/F32 weights (1,392 Q8_0 + 1,181 F32 tensors)
- 62.23 B parameters, 100% active per token
Intended use
This model is intended as a base for fine-tuning and Reinforcement Learning. It is recommended to be run inside the Prime Agent harness.
Usage
hf download HaithamalWaisy/kimi-k3-merged kimi-k3-merged-2exp.gguf --local-dir .
License
Apache 2.0 (inherited from Kimi K3). Quality was not benchmarked by this project.