K2-Horizon-7B-Uno MLX SLQ 4.5bpw

Non-uniform SLQ quantization of the merged IFM/K2-Horizon-7B-Uno model — a diffusion-augmented LLM based on K2-Horizon-7B. The LoRA adapter is baked into the base weights, so it runs as a standard autoregressive model.

Upstream model: IFM/K2-Horizon-7B-Uno by Institute of Foundation Models, released under Apache 2.0.

Conversion: Quantized to MLX format using Hermes Agent with SLQ-MLX (per-layer greedy allocator, 4.500 bpw target).

Quickstart

pip install -U mlx-lm

python3 -m mlx_lm.generate \
  --model hermitdave/K2-Horizon-7B-Uno-MLX-slq-4.5bpw \
  --prompt "Explain step by step." \
  --max-tokens 512 --temp 1.0 --top-p 0.95

This checkpoint uses a custom architecture file (slq_model.py) that wraps the base model with per-layer QuantizedLinear layers. Loading it requires trust_remote_code=True.

Reasoning

K2-Horizon-7B is a reasoning model. Always use reasoning_effort="high":

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="hermitdave/K2-Horizon-7B-Uno-MLX-slq-4.5bpw",
    messages=[{"role": "user", "content": "Explain step by step."}],
    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
)
print("Reasoning:", getattr(response.choices[0].message, "reasoning_content", None))
print("Answer:", response.choices[0].message.content)

Analysis

Full 2–8 sweep, all 36 blocks GPTQ-calibrated on 8,192 FineWeb-Edu tokens (4 × 2048).

Zero-shot recovery (100 examples/task)

Mean recovery 0.968 — NOT task-lossless (paper threshold ≥ 0.99). The 95% CIs all cross zero at n=100, so the degradation is not statistically significant on this sample, but the mean is below the paper’s threshold.

Citation

@misc{k2_horizon_7b_uno,
  title        = {K2-Horizon-7B-Uno},
  author       = {Institute of Foundation Models},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/IFM/K2-Horizon-7B-Uno}},
}

License

Apache 2.0 (same as upstream).