LoCo-7B: Think Less, Locally

TL;DR: LoCo compresses a reasoning model's chain of thought using its own traces. It locally detects overthinking states, pairs them with shorter verified continuations regenerated from the same states, and applies DPO to these state conditioned pairs to teach the model when to stop thinking.

LoCo-7B is a reasoning-compressed version of DeepSeek-R1-Distill-Qwen-7B. Completion length is reduced by 41.7–45.4% on mathematics and 23.4–36.3% on code, while accuracy is maintained or improved.

Usage

from vllm import LLM, SamplingParams

model = LLM(model="MuyaoYuan/LoCo-7B", max_model_len=36864, dtype="bfloat16")
tok = model.get_tokenizer()
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Your math problem here. Please reason step by step, and put your final answer within \\boxed{}."}],
    tokenize=False,
    add_generation_prompt=True,
)
output = model.generate([prompt], SamplingParams(temperature=0.6, top_p=0.95, max_tokens=32768))
print(output[0].outputs[0].text)

The full code for data construction, training, and evaluation is available on GitHub.

License

MIT