K2-Horizon-MoVA-36B-A4B · EXL3

EXL3 quantizations of IFM's K2-Horizon-MoVA-36B-A4B, one folder per bitrate. Every pack is scored against the BF16 original on held-out text before it is uploaded.

bpw	size	smallest card	top-1 vs BF16	mean KLD	p99 KLD	download
8.00	38.07 GB	96 / 48 GB	96.43% (9,874 / 10,240)	0.0047	0.042	8.00bpw
6.50	31.34 GB	48 GB	90.68% (9,286 / 10,240)	0.0335	0.336	6.50bpw
5.00	24.56 GB	32 GB	85.83% (8,789 / 10,240)	0.0923	0.826	5.00bpw
4.00	20.05 GB	24 GB	84.81% (8,685 / 10,240)	0.1082	0.909	4.00bpw
2.50	13.27 GB	16 GB	83.71% (8,572 / 10,240)	0.1326	1.153	2.50bpw
2.00	11.01 GB	8 GB + offload	81.66% (8,362 / 10,240)	0.1604	1.322	2.00bpw

top-1: positions where the pack's most likely next token matches the reference's, counted over 10,240 positions of held-out text.
KLD: KL(reference ‖ pack) per position over the same positions, mean and 99th percentile. Lower is closer.
Reference: IFM's own implementation (modeling_k2_horizon.py) running the original BF16 weights with fp32 activations.
For scale: the unquantized BF16 weights running in the same exllamav3 code match the reference on 97.71% of positions (10,006 / 10,240) with mean KLD 0.0030. That is about the best any pack here can score.
size is the whole folder on disk. smallest card is the smallest GPU that holds it resident with room for KV cache. The 8.00 pack is 38 GB, so it runs on a 48 GB card as well as a 96 GB one.
The 2.00 row is a floor pack: about 11 GB of weights, so an 8 GB card needs expert offload.

About these quants

Made with SAGE, our own mixed-precision quantization method for EXL3. The number in each folder name is the average bitrate of the model body. The output head is 6-bit in every pack.
The scores compare each pack with the reference above on text that was not used to make the quants. They measure how closely a pack tracks BF16, not task accuracy.

What K2-Horizon-MoVA is

K2-Horizon-MoVA-36B-A4B is the sparse member of IFM's K2-Horizon family. It stores 36B parameters but runs about 4B per token, using two kinds of routing: a Mixture-of-Experts feed-forward block and Mixture-of-Values attention (MoVA), where each token's value projection is itself routed through a bank of experts. It is Apache-2.0, and IFM is publishing the training data, logs, intermediate checkpoints and code alongside the weights.

Highlights from IFM's model card:

Frontier-class results at 4B active parameters. IFM reports that on agentic and reasoning benchmarks it outscores open dense models of about 30B and MoE models up to 15× its size.
512K context. Native 524,288-token context from midtraining onward.
Intermediate checkpoints. Every training stage is published as a branch of the base repo (pretrain_*, mid_1_* … mid_4_*, sft_1_*, sft_2_*), so you can study how capability changes across training.
Fully open. Pretraining and midtraining data (IFM/K2-Horizon-Pretrain-Data, IFM/K2-Horizon-Midtrain-Data), training logs and a blog post are out. The code repository and technical report are listed for end of September 2026.

Architecture

From the model's config.json:


Layers	48 decoder layers, hidden size 2,560
Attention	32 query heads, 8 KV heads (GQA), head dim 128, RoPE θ = 10,000,000, softplus output gate
MoVA values (layers 3–47)	64 value experts, 4 active per token, sigmoid router
MoE feed-forward (layers 3–47)	100 routed experts (width 768) + 1 shared expert, 8 active per token, sigmoid router, selection-only router bias, normalized weights × 2.5
Dense layers (0–2)	standard attention + SwiGLU MLP (width 6,144)
Norm	grouped RMSNorm (2 groups), ε = 1e-6
Vocabulary	250,624 tokens, untied input/output embeddings
Context	524,288 tokens

Training

IFM trained it in stages, each continuing from the previous one (about 25.1T tokens in total):

Stage	Tokens	Sequence length	Purpose
Pretraining	22.9T	8K	Pretraining
Midtraining 1	1.1T	32K	Context extension
Midtraining 2	498B	128K	Context extension
Midtraining 3	110B	512K	Context extension
Midtraining 4	199B	512K	Context extension, mix shifted toward agentic and reasoning data
SFT 1	219B	512K	Broad domain coverage
SFT 2	50B	512K	High-quality subset of SFT 1, learning-rate decay

Reported results

Scores for the BF16 model from IFM's card (%, high reasoning effort, Artificial Analysis categories). They were not re-run on these quants. Use the top-1/KLD columns above to see how closely each pack tracks BF16.

Benchmark	What it tests	K2-Horizon-MoVA-36B-A4B
τ³-Banking	Agentic tool use	26.8
Terminal-Bench 2.1	Agentic terminal use	58.6
SciCode	Scientific coding	38.9
Humanity's Last Exam (no tools)	Expert-level reasoning	25.2
GPQA Diamond	Graduate-level science QA	80.8
CritPt	Frontier physics reasoning	2.1
AA-LCR	Long-context reasoning	66.3
AA-Omniscience	Factual accuracy	18.8
AA-Omniscience	Non-hallucination rate	69.2

Using it well

IFM's recommended settings, which apply to these packs as well:

Reasoning effort high on every request: chat_template_kwargs: {"reasoning_effort": "high"}. All of IFM's reported results use it.
Sampling: temperature=1.0, top_p=0.95.
Thinking comes back in reasoning_content and the answer in content when your server uses a K2-Horizon reasoning parser.
Tool calls can use json, xml (default) or xml_typed, selected with chat_template_kwargs: {"tool_call_format": ...}.
Each folder ships the original chat_template.jinja and tokenizer.

Running

These packs load in exllamav3 with the K2-Horizon architecture port (K2HorizonForCausalLM: MoVA values, softplus attention gate, grouped RMSNorm, selection-only router bias). Upstream exllamav3 does not include this architecture yet. The port and a serving recipe with RTX PRO 6000 speed numbers will be linked here when they are published.

License and credit

Apache-2.0, same as the base model. The model, data and training are IFM's work. See the original model card for full details.

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}

vcruz305/K2-Horizon-MoVA-36B-A4B-EXL3