On this repo...

K2-Horizon-7B quantized with an imatrix generated from open-perfectblend.

The IQ4-XS quant performs slightly better than the official Q4_K_M varian, with lower perplexity, higher accuracy, faster inference and smaller model size. The other i-quants will be measured against their published variants in the future, but you can find a comparison of the quants included in this repo below.

3 Imatrix files were generated by going over corpuses with different token counts: 1M, 4M and 8M tokens. The published variant uses the 8M token imatrix.

The graphs below were generated with python with AI assistance to obtain more visually appealing results. The data was obtained after evaluating the models on wikitext, winogrande, hellaswag and running llama-bench.

IQ4-XS vs published Q4_K_M vs Q8_0 baseline

I-Quants comparison

---

K2-Horizon-7B-GGUF

[!NOTE] This repository contains GGUF versions of the IFM/K2-Horizon-7B for use with llama.cpp.

Multiple precision and quantization variants are available, including BF16, Q8_0, Q6_K, Q5_K_M, Q5_0, and Q4_K_M. All GGUF files include tokenizer metadata and a llama.cpp-compatible chat template.

[!WARNING] 💡 Important Deployment Notes: These models require a version of llama.cpp containing K2 Horizon architecture support. PR to llama.cpp is in progress. MBZUAI-IFM fork of llama.cpp is in https://github.com/MBZUAI-IFM/llama.cpp/tree/model/K2Horizon

K2-Horizon-7B is the medium dense member of the K2-Horizon family: a 7B-core decoder-only model with a 512K context window.

K2-Horizon-7B Highlights

• Strong dense baseline. A 7B-class dense model evaluated across agentic, coding, long-context, and reasoning benchmarks.
• 512K context. Native 524,288-token context from the midtraining stages onward.
• Intermediate checkpoints. Intermediate checkpoints are released so capability changes can be studied across training rather than at a single checkpoint.
• Fully open. Training data and recipe, training code, and evaluation resources are public.

Benchmark Results

The chart at the top of this card shows K2-Horizon-7B against selected reference models. The table below lists every comparison model used in the figure.

Full Results

Scores in %. Bold marks the best score in each row. BrowseComp: our model uses the Discard-all@95k context-length protocol proposed in the DeepSeek-V3.2 technical report; comparison models may use different harnesses.

GGUF BF16 vs. Quantized

The evaluation context length is set to 65,536 tokens. Unless otherwise specified, all tasks are evaluated in a 0-shot setting. The GGUF models have currently been evaluated only on non-agent tasks. Results for agent tasks will be released later.

Quickstart

Serving

vLLM, recipe at recipes.vllm.ai/IFM:

vllm serve IFM/K2-Horizon-7B \
  --trust-remote-code \
  --dtype bfloat16 \
  --max-model-len 131072 \
  --tensor-parallel-size 1 \
  --reasoning-parser k2_horizon \
  --enable-auto-tool-choice \
  --tool-call-parser k2_horizon

SGLang, this is the recipe validated in the SGLang K2 Horizon cookbook:

sglang serve \
  --model-path IFM/K2-Horizon-7B \
  --revision 69ada542b68fe13d767479db2ab9421baff88681 \
  --tp 1 \
  --dtype bfloat16 \
  --attention-backend fa3 \
  --reasoning-parser k2_horizon \
  --host 0.0.0.0 \
  --port 30000

API Usage

[!Tip] Recommended settings: reasoning_effort="high", temperature=1.0, top_p=0.95, and at least 32,768 output tokens. Reasoning depth is selected per request through chat_template_kwargs. Thinking is returned in reasoning_content and the answer in content.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="IFM/K2-Horizon-7B",
    messages=[{"role": "user", "content": "Explain the result step by step."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high", "tool_call_format": "xml"}},
)
message = response.choices[0].message
print("Reasoning:", getattr(message, "reasoning_content", None))
print("Answer:", message.content)

Our model supports multiple tool calls formats, which can be changed with chat_template_kwargs. The supported values are json, xml, and xml_typed . The default is xml. Keep --tool-call-parser k2_horizon enabled to parse the selected format.

Transformers

Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "IFM/K2-Horizon-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
)

inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
inputs.pop("token_type_ids", None)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Best Practices

1. Reasoning effort: always high. All reported results use high reasoning effort. Pass {"chat_template_kwargs": {"reasoning_effort": "high"}} on every request; medium and low trade accuracy for speed and are not recommended for evaluation.
2. Sampling parameters. temperature=1.0, top_p=0.95.
3. Output length. Allow at least 32,768 output tokens so reasoning is never cut off. Truncated reasoning is a failed response, not a shorter one.
4. Serving. Use the validated SGLang recipe above: BF16, TP=1, FlashAttention-3. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the SGLang cookbook.
5. Parsers. Enable the k2_horizon reasoning parser for chat, and add the k2_horizon tool-call parser for agent use. Leave both off for plain completion-style generation.
6. Revisions. Pin a revision tag when reproducibility matters. main is the default checkpoint; base_final and the mid_*_final tags identify training stages.

Citation

@misc{k2horizon2026,
  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
  author = {{IFM Team}},
  year   = {2026},
  url    = {https://ifm.ai/blog/k2/},
}