Ultra-compact 0.5B parameter agentic model fine-tuned for high-precision JSON tool and function calling, tested on unseen tool schemas.
transformers, sentence-transformers, peft, and standard inference pipelines.Evaluated on the official held-out test split under standard production evaluation protocols:
| Benchmark Metric | Score / Value |
|---|---|
| test_loss | 0.7136 |
| test_perplexity | 2.0413 |
| baseline_test_loss | 2.0568 |
| baseline_test_perplexity | 7.8200 |
| perplexity_reduction | -73.9% |
💡 Benchmark Takeaway: Fine-tuning establishes substantial gains over the zero-shot base checkpoint, dramatically suppressing false predictions and improving domain confidence.
Install the required dependencies:
pip install transformers torch sentence-transformers peft accelerate
import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_repo = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_repo = "vennyvirtuoso/qwen2.5-0.5b-tool-calling-hermes"
# 1. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(adapter_repo)
base_model = AutoModelForCausalLM.from_pretrained(
base_repo,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# 2. Attach LoRA Adapter
model = PeftModel.from_pretrained(base_model, adapter_repo)
model.eval()
# 3. Define Tools & Prompt
tools = [
{
"type": "function",
"function": {
"name": "get_stock_price",
"description": "Retrieve current price and daily volume for a given ticker symbol.",
"parameters": {
"type": "object",
"properties": {"ticker": {"type": "string", "description": "Stock symbol (e.g. AAPL)"}},
"required": ["ticker"]
}
}
}
]
messages = [
{"role": "system", "content": f"You are a helpful agent with access to functions.\n<tools>{json.dumps(tools)}</tools>"},
{"role": "user", "content": "What is the stock price of NVDA right now?"}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.1)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
<tool_call>
{"name": "get_stock_price", "arguments": {"ticker": "NVDA"}}
</tool_call>
| Parameter | Specification |
|---|---|
| Base Foundation Model | Qwen/Qwen2.5-0.5B-Instruct |
| Fine-Tuned Checkpoint | vennyvirtuoso/qwen2.5-0.5b-tool-calling-hermes |
| Task Category | Agentic Function & Tool Calling |
| Training Dataset | NousResearch/hermes-function-calling-v1 |
| Precision | Mixed Precision (bfloat16 / fp16) |
| License | Apache 2.0 (Permissive Commercial & Research) |
This model is open-sourced under the Apache 2.0 license. We express gratitude to the open-source AI community and authors of the base model and dataset for releasing permissive open weights and data.
Qwen/Qwen2.5-0.5B-Instruct