K2-Horizon-0.9B-System1
IFM/K2-Horizon-0.9B is a reasoning model: it writes a long <ifm|think> block before every answer, and its card asks for 32K output tokens. This fine-tune is the System 1 version. It answers right away, with no think block, and shows a line or two of working only when the task needs it. On math it uses about 5× fewer tokens than the base model in no-think mode at the same MATH-500 accuracy.
The original thinking modes are still there. reasoning_effort now accepts none (the new default), low, medium and high.
| Variant | Repo |
|---|---|
| Safetensors (this repo, same layout and remote code as the base) | Lucasgitguy-co/K2-Horizon-0.9B-System1 |
| MLX 8-bit / 4-bit | -MLX-8bit · -MLX-4bit |
| GGUF Q8_0 / Q4_K_M (llama.cpp, Ollama, LM Studio) | -GGUF |
| Training data | k2-horizon-system1-distill |
Results
All rows are the same model family in no-think mode (reasoning_effort="none"): the base model with an empty think block prefilled, and this fine-tune. 684 held-out prompts, temperature 0.6, top-p 0.95, 2,048-token cap, one sample each.
| GSM8K (200) | MATH-500 (100) | HumanEval+ (164) | IFEval strict (200) | Chat (20) | |
|---|---|---|---|---|---|
| K2-Horizon-0.9B, no-think | 68.5 | 45.0 | 51.8 | 52.0 | 100 |
| K2-Horizon-0.9B-System1 | 71.5 | 45.0 | 48.2 | 47.0 | 100 |
Median output tokens:
| GSM8K | MATH-500 | HumanEval+ | IFEval | Chat | |
|---|---|---|---|---|---|
| K2-Horizon-0.9B, no-think | 322 | 1,580 | 219 | 105 | 36 |
| K2-Horizon-0.9B-System1 | 183 | 317 | 161 | 215 | 113 |
The untrained base in no-think mode doesn't really answer fast. On math it just moves its reasoning into the visible answer, leaking <ifm|think> tags in 72 of 684 answers, and 9.5% of answers hit the 2,048-token cap. After training, leaks drop to 3 and truncations to 2.9%.
System 2 check (high effort, 30 math prompts, 6,144-token cap): thinking still works. Every answer still has a think block. GSM8K is 14/15 for both base and fine-tune. MATH-500 is 12/15 base vs 11/15 fine-tune, which is within noise on 15 items. Median output was 970 tokens base vs 857 fine-tune, and 1 vs 3 answers hit the cap. On hard problems high effort is much stronger than System 1, so keep using it there.
How the numbers were measured- GSM8K: random 200 from the test split. MATH-500: random 100 of HuggingFaceH4/MATH-500. Answers are checked with
math-verifyon the last\boxed{}. - HumanEval+: all 164 evalplus tasks; the extracted code is run against the plus tests.
- IFEval: random 200 of google/IFEval, scored with the official google-research checker, strict prompt-level accuracy.
- Chat: 20 everyday prompts, counted as a pass if the answer is non-empty and not a repetition loop.
- Scoring code and raw outputs:
pipeline/in this repo.
These are small samples. A 100-item MATH-500 slice has roughly ±5 points of noise.
How it was made
This is self-distillation of System 2 into System 1, following Distilling System 2 into System 1 (Yu et al., 2024) plus Qwen3-style thinking-mode fusion. Everything ran on one MacBook Air M4 with 16 GB, using MLX.
- Port and check. With this config the k2_horizon code path is plain Llama (YaRN ×16 RoPE). It was loaded in mlx-lm, and its logits match the original Transformers remote code in fp32: max |Δ| 2.4e-4 on a 16.8K-token prompt, 100% top-1 agreement (
pipeline/parity.json). - Teacher. The base model answered train-split prompts at
reasoning_effort="high"with up to 8K thinking tokens. Only verified answers were kept: math checked against ground truth, code passing its tests, instruction-following passing the constraint checkers. Truncated thinking was dropped. - Targets. About 85% of examples are System 1: the prompt, then an empty think block, then the teacher's final answer (the part after thinking, so brief working plus the result). About 15% keep the full thinking, so high effort still works.
- LoRA. Rank 16, scale 2, on all attention and MLP projections of all 28 layers. lr 1e-4, cosine schedule, 2 epochs, loss on assistant tokens only. Fused into the weights afterwards.
The release uses the 142-example set (134 train, 8 validation, mostly math). Two other variants scored worse on average and were not released:
- adding 1,000 self-distilled no-think instruction/code/chat answers: MATH-500 fell to 33 because answers got too terse;
- a 50/50 weight merge of the two: in between, on average.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "Lucasgitguy-co/K2-Horizon-0.9B-System1"
tok = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="bfloat16", trust_remote_code=True)
msgs = [{"role": "user", "content": "What is 17*23?"}]
inputs = tok.apply_chat_template(msgs, add_generation_prompt=True, return_dict=True, return_tensors="pt")
inputs.pop("token_type_ids", None)
out = model.generate(**inputs, max_new_tokens=512, do_sample=True, temperature=0.6, top_p=0.95)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
# reasoning_effort="high" in apply_chat_template switches back to full thinking
MLX:
pip install mlx-lm
mlx_lm.generate --model Lucasgitguy-co/K2-Horizon-0.9B-System1-MLX-8bit --prompt "What is 17*23?"
vLLM and SGLang: use the same recipes as the base model card. With the k2_horizon reasoning parser, no-think answers come back with an empty reasoning_content.
The model needs Transformers 5.x for the remote code (the base card validated 5.14–5.16). Generation should stop on <|ifm|im_end|> (id 64019) as well as <|endoftext|>; generation_config.json already lists both.
Limitations
- Trade-off. Instruction-following (IFEval −5) and code (HumanEval+ −3.6) are slightly below the base in no-think mode, and chat answers are a bit longer. The training set is small and mostly math. More verified non-math teacher data is the obvious next step.
- Hard problems. For contest math, the full thinking mode (
reasoning_effort="high") is still far stronger. System 1 is for quick answers. - Inherited. All limitations of K2-Horizon-0.9B carry over: it can be wrong, biased or unsafe, and should be evaluated before you rely on it.
Credits and licence
- Base model: IFM/K2-Horizon-0.9B (Apache-2.0) by the IFM team. Its configuration and modeling code are included unchanged.
- Method: Yu et al., Distilling System 2 into System 1 (2024); the Qwen3 technical report (thinking-mode fusion).
- Tools: MLX / mlx-lm, llama.cpp, math-verify.
- Verifiers: the IFEval checker from google-research and
if_functions.pyfrom allenai/open-instruct (both Apache-2.0). - Prompt sources and their licences are listed in the dataset card.
Released under Apache-2.0, like the base model.