LFM2.5-8B-A1B-Claude-Opus4-6-Reasoning

LiquidAI's LFM2.5-8B-A1B fine-tuned on Claude 4.6 Opus reasoning traces.

This model adds explicit step-by-step chain-of-thought reasoning to LFM2.5-8B, distilling the reasoning style of Claude 4.6 Opus into an efficient 8B model that runs fully locally on Apple Silicon. Inspired by Jackrong's reasoning distillation pipeline; adapted for MLX and Apple Silicon using KL-regularized LoRA.

---

What This Model Does

Before answering, the model produces an explicit reasoning trace inside <think> tags. The reasoning trace makes the logic visible and inspectable, and forces the model to actually work through problems rather than pattern-match to a quick guess.

The base LFM2.5-8B model answers immediately. This model first thinks through the problem step-by-step — like Claude does — then gives the answer.

---

Model Details

---

Training Details

The Problem: Reasoning Collapse

Standard SFT on reasoning data causes reasoning collapse: the model learns the <think> format but produces shallow, 46-word think-blocks and gets worse at actual problem solving. In our experiments, a naive SFT run was beaten by the unmodified base model 71% of the time.

The Fix: KL-Regularized LoRA

This model uses a combined loss that simultaneously teaches the Claude reasoning format while penalizing distributional drift from the base model:

total_loss = CE_loss + kl_weight * KL(p_finetuned || p_base)

The KL term prevents reasoning shortcuts. Base logits are computed with a scale=0 LoRA trick — no second model copy needed in memory.

LoRA Configuration

Training Hyperparameters

Training Data

~12,000 samples combining three datasets of Claude 4.6 Opus reasoning traces:

All assistant turns normalized to <think>{reasoning}</think>{answer}. System messages stripped.

---

Evaluation

Evaluated via LLM-as-a-judge (meta-llama/llama-3.3-70b-instruct via OpenRouter) on 24 questions spanning GSM8K math, MATH500 hard problems, logic puzzles, coding, and open-ended questions. A/B answer order randomized per question to prevent position bias.

Win Rate vs Base Model

KL regularization cut the base model's advantage in half. 65% of questions are now a tie or a win for this model.

Reasoning Depth

By category (vs base model):

• GSM8K: essentially equal (3 base wins, 3 model wins, 6 ties)
• Open-ended: mostly tied (2 base, 0 model, 5 ties) — good general capability
• MATH500 hard: base still ahead (3 base, 1 model) — hard math remains a stretch

---

Usage

mlx-lm (Apple Silicon, recommended)

pip install mlx-lm

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("sahilchachra/LFM2.5-8B-A1B-Claude-Opus4-6-Reasoning")

messages = [{"role": "user", "content": "What is 15% of 240?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

response = generate(
    model, tokenizer,
    prompt=prompt,
    max_tokens=2048,
    sampler=make_sampler(temp=0.6),
    verbose=True
)

Transformers (CPU/GPU)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "sahilchachra/LFM2.5-8B-A1B-Claude-Opus4-6-Reasoning"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)

messages = [{"role": "user", "content": "Explain why 0.999... equals 1."}]
inputs = tokenizer.apply_chat_template(messages, return_tensors='pt', add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=2048, temperature=0.6, do_sample=True)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

The model uses ChatML format. Always use apply_chat_template.

---

Limitations

• Hard math: Still loses to the base model on MATH500 Level 4-5 problems. Reasoning traces are deeper but not always correct on competition-level mathematics.
• Seq length: Training used 2,048 token max; some long reasoning traces were truncated. Very long multi-step problems may get cut off.
• English-primary: Training data is overwhelmingly English. The base model supports 9 languages but reasoning fine-tuning was English-only.
• Not RLHF'd: SFT + KL regularization only — no reinforcement learning from human or AI feedback.

---

Technical Note

The key memory challenge: KL regularization requires base model logits at every training step, but two 8B model copies exceed 24GB unified memory. Solved using the LoRA scale trick:

# Pass 1: scale=0 → model behaves as frozen base (no second copy needed)
for layer in lora_layers:
    layer.scale = 0.0
base_logits = model(inputs)   # no gradient

# Pass 2: restore scale → fine-tuned forward with gradients
for layer, s in zip(lora_layers, orig_scales):
    layer.scale = s
ft_logits = model(inputs)     # gradient flows through LoRA only

Full training code: train_kl.py

---

Citation

If you use this model, please also credit:

• Base model: LiquidAI/LFM2.5-8B-A1B
• Distillation inspiration: Jackrong/Qwen3.5-27B-Claude-4.6-Opus-Reasoning-Distilled
• LoRA: Hu et al., 2021 — arXiv:2106.09685

---

Everything — training, inference, and benchmarking — was done on a Mac Mini M4 Pro (24GB unified memory) using MLX. No cloud GPUs, no A100s. Serious reasoning distillation is achievable on consumer Apple Silicon hardware.