TinyToolRouter-0.6B

A lightweight, CPU-first tool router fine-tuned from Qwen3-0.6B.

GitHub: @roboroyce

TinyToolRouter-0.6B is a 0.6-billion-parameter language model designed to select tools from a supplied list of tool schemas. Given a natural-language request and the available tools, it produces exactly one structured JSON tool call or explicitly abstains when no tool applies.

Example

Input request: What's the weather in Oslo?

Output:

{
  "name": "get_weather",
  "arguments": {
    "city": "Oslo"
  }
}

When no listed tool applies, the model abstains:

{
  "name": "no_tool",
  "arguments": {}
}

Key features

• Small footprint: 0.6B parameters, suitable for CPU-first inference.
• Schema-aware routing: Selects from tool definitions supplied at inference time.
• Structured output: Produces a single JSON object representing a proposed tool call.
• Explicit abstention: Supports a dedicated no_tool response.
• Paraphrase robustness: Evaluated on requests phrased differently from the training examples.
• Unseen-schema evaluation: Tested on tools absent from the training set.
• Multiple formats: Safetensors and GGUF builds are available in this repository.

Model details

File sizes are approximate and may vary slightly by build.

Benchmark results

TinyToolRouter was evaluated on a held-out set of 231 cases covering three distinct evaluation families.

Overall performance

Exact match improved by 40.26 percentage points over the base model.

Performance by evaluation family

What these results mean

• Paraphrase robustness: Exact match increased from 22.90% to 92.37% on requests that use unseen phrasing for tools encountered during training.
• Unseen tool schemas: Exact match increased from 85.00% to 86.67% on tools absent from the training set. This is a modest improvement, not evidence of fully general tool-schema understanding.
• Abstention: The fine-tuned model correctly abstained on all 40 held-out no_tool cases, with 97.56% measured precision.

Evaluation methodology

Both models were evaluated using identical prompts and generation settings:

• do_sample=False
• max_new_tokens=128
• Held-out evaluation set: 231 cases
• Evaluation metric: exact match of the expected structured output

The benchmark separates familiar tool schemas with paraphrased requests, genuinely unseen tool schemas, and requests that should not trigger a tool call.

The reported metrics are from the safetensors model. The GGUF conversions have not been independently benchmarked, so equivalent performance across formats should not be assumed.

Quickstart

Install dependencies

pip install torch transformers

Run with Transformers

import json
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Robokid1134/TinyToolRouter-0.6B"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
model.eval()

tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city.",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {"type": "string"}
            },
            "required": ["city"],
        },
    }
]

system_prompt = (
    "You are a tool router. You are given a list of available tools "
    "and a user request. Reply with exactly one JSON object of the form "
    '{"name": "<tool_name>", "arguments": {"<argument>": "<value>"}}. '
    "Use only the listed tools and their declared arguments. "
    'If no listed tool applies, reply {"name": "no_tool", "arguments": {}}. '
    "Output JSON only, with no additional prose."
)

messages = [
    {"role": "system", "content": system_prompt},
    {
        "role": "user",
        "content": json.dumps({
            "tools": tools,
            "request": "What's the weather in Oslo?"
        }),
    },
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(prompt, return_tensors="pt")

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
    )

new_tokens = output[0][inputs["input_ids"].shape[1]:]
response = tokenizer.decode(
    new_tokens,
    skip_special_tokens=True,
)

print(response)

Expected output:

{"name": "get_weather", "arguments": {"city": "Oslo"}}

Note: enable_thinking=False is recommended for direct routing output. The example uses the Qwen3 chat template and assumes a Transformers version that supports this argument.

GGUF: CPU inference

GGUF builds are available in the repository's gguf/ directory.

Run a GGUF model using llama.cpp:

llama-cli \
  -m gguf/model-Q4_K_M.gguf \
  -p "<your prompt>"

The prompt should follow the model's expected chat template and include the available tool schemas and user request.

For repeated use, a dedicated prompt template or a compatible Ollama model configuration can simplify integration.

Benchmark caveat: The GGUF files were converted from the merged fine-tuned model but have not been independently re-evaluated. The benchmark figures above apply to the safetensors model.

Tool schema format

The model expects a JSON object containing a list of available tools and a natural-language request.

{
  "tools": [
    {
      "name": "get_weather",
      "description": "Get the current weather for a city.",
      "parameters": {
        "type": "object",
        "properties": {
          "city": {
            "type": "string"
          }
        },
        "required": ["city"]
      }
    }
  ],
  "request": "What's the weather in Oslo?"
}

The schema format is JSON-Schema-style, with a name, description, and parameters object for each tool.

Training covered the following argument types:

• string
• integer
• number
• boolean
• array of strings
• array of numbers

Examples included required and optional arguments, along with distractor tools that should not be selected.

The model returns a proposed tool name and its arguments. It does not execute the tool or verify that the underlying action succeeds.

Training

The LoRA adapter was merged into the base model for deployment.

Training loss was computed on the assistant's JSON output, with prompt tokens masked from the loss.

Dataset

The dataset consists of original synthetic examples authored for this project and released under Apache-2.0.

No scraped text, chat logs, or personal data were used. Slot values are synthetic, including example email addresses, dates, and city names.

Tool coverage

Tools represented in training (9):

• get_weather
• send_email
• create_calendar_event
• search_web
• convert_currency
• set_timer
• translate_text
• filter_items
• tag_note

Unseen tools reserved for evaluation (5):

• play_music
• get_stock_price
• book_restaurant
• draw_chart
• create_reminder

Data separation

The training and held-out sets use disjoint templates, value pools, and example IDs.

The unseen_schema evaluation family contains tools absent from training entirely. The training set also includes abstention examples, while the held-out no_tool requests are disjoint from those training examples.

These design choices help test whether the model learned useful routing patterns rather than simply memorizing identical examples. They do not eliminate every possible source of dataset bias or leakage.

Limitations

• English only: The model was trained and evaluated on English-language examples.
• Limited tool coverage: Training covers nine tool schemas, with five additional schemas used for evaluation.
• Synthetic data: Performance on real-world agent requests may differ from the reported benchmark.
• Modest unseen-schema gains: The improvement on unseen schemas is only 1.67 percentage points.
• Limited argument extraction: Arguments often appear explicitly in the request. The evaluation primarily tests tool selection, argument correctness, and structured output rather than unrestricted information extraction.
• Small evaluation set: Results are based on 231 held-out examples. Small differences may reflect only a few cases.
• Single random seed: Multi-seed variance has not been measured.
• Quantized models not re-benchmarked: The reported metrics do not establish equivalent performance for the GGUF builds.
• Context length: Training used a maximum sequence length of 512 tokens. Long tool lists may exceed the model's familiar input distribution or be truncated by an inference pipeline.

Intended use

TinyToolRouter-0.6B is intended for:

• Local AI agents running on CPU-only or low-VRAM machines.
• Lightweight tool-selection front ends for larger language models.
• Routing between a predefined set of application tools.
• Explicitly declining requests that do not match any available tool.
• Research into compact language models, tool calling, and structured generation.
• Reproducible experiments comparing a small base model against a narrowly fine-tuned router.

Out of scope

This model is not a general-purpose assistant. It is not intended to independently answer general questions, execute tools, or make safety-critical decisions.

Do not rely on its output alone to authorize sensitive actions or execute shell commands. The calling application should validate the selected tool, enforce an allowlist, validate arguments against the actual schema, and require appropriate authorization before executing an action.

Citation

If you use this model in research or another project, you can cite it as follows:

@misc{tinytoolrouter06b,
  title  = {TinyToolRouter-0.6B: A CPU-First, Schema-Strict Tool Router},
  author = {Robokid1134},
  year   = {2026},
  note   = {LoRA fine-tune of Qwen/Qwen3-0.6B},
  url    = {https://huggingface.co/Robokid1134/TinyToolRouter-0.6B}
}

Attribution and license

TinyToolRouter-0.6B is fine-tuned from Qwen/Qwen3-0.6B, which is released under the Apache-2.0 license.

This derivative model is distributed under Apache-2.0, with the applicable upstream license and copyright notices preserved. The project's original code and generated dataset are also released under Apache-2.0.