Qwen2.5-0.5b-Next

An experimental Qwen2.5-0.5B-Instruct hybrid with 12 GQA layers and 12 local + compressed-memory attention layers. Recovered with 200 steps of chat training on No Robots.

Keeps 128 recent tokens exact and averages older tokens in chunks of 16, using a real compressed KV cache. Recovery context: 2,048 tokens. Longer contexts up to 8,192 were explored in small custom tests.

An independent community experiment, not an official Qwen release. Use the included loader to activate hybrid attention.

Inference

Install torch>=2.5, transformers>=4.51,<6, huggingface_hub, and safetensors, then:

import runpy
from huggingface_hub import hf_hub_download

load = runpy.run_path(hf_hub_download("Ma7ee7/Qwen2.5-0.5b-Next", "inference.py"))["load"]
chat = load("Ma7ee7/Qwen2.5-0.5b-Next")
chat("Hello!")

History persists between calls. chat.reset() clears it. Generation stops at the end-of-turn token or available context budget.

chat("What did I just ask?")
chat("Explain gravity.", max_new_tokens=512, temperature=0.7)

Default total context is 2,048 tokens. For experimental longer inference, use load("Ma7ee7/Qwen2.5-0.5b-Next", max_context=8192).