Qwen2.5-AlphaCoder-14B

An experimental fine-tune of Qwen2.5-Coder-14B-Instruct that explores lightweight distillation: transferring the agentic, tool-calling behavior of Qwen3-32B into a smaller Qwen2.5 model.

Method

• Teacher data: 500 Qwen3-32B tool-use trajectories from the Qwen3 subset of Toucan-1.5M (multi-turn, real MCP tool calls)
• Training: QLoRA (r=16, alpha=32), 2 epochs, loss on assistant tokens only
• Context: whole trajectories up to 2,048 tokens, no truncation
• Format: Qwen2.5-native <tool_call> / <tool_response> tags

Files

• Root: 4-bit (bitsandbytes) model weights
• adapter/: the QLoRA adapter on its own

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "Rumiii/Qwen2.5-AlphaCoder-14B"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, device_map="auto")  # needs bitsandbytes + GPU

# pass `tools=[...]` to tok.apply_chat_template for function calling

Limitations

This is a small proof-of-concept run (500 samples). It has not been benchmarked, and it should not be expected to match the capability of the 32B teacher.

License

Apache 2.0, following the base model and dataset.