mini-agent-v2-GGUF

A 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct, shipped as merged bf16 weights and six GGUF quants.

bf16 weights · GGUF quants

checkpoint b15 (d1497d77e5) · evaluated 2026-10-04

---

Summary

mini-agent-v2 is a QLoRA fine-tune of Qwen2.5-Coder-1.5B-Instruct, trained jointly on function calling, Bash and Python on a single 8 GB consumer GPU. It is small enough to run on a laptop. It ships as merged bf16 weights and as GGUF builds from Q3_K_M to bf16.

It speaks the standard tool-call protocol: Qwen's own chat template with <tool_call>{"name": ..., "arguments": {...}}</tool_call>. That is the format llama.cpp (--jinja), vLLM, Ollama and OpenAI-style clients already parse. You don't need a custom system prompt or parser: pass your tools in the normal tools field. On the Berkeley Function Calling Leaderboard (BFCL) questions it answers 86.2% correctly, up from 52.7% for the base model.

At a glance

Specifications

---

Results

These are the three tasks the model was trained for, compared with the base model under an identical harness. Every answer is executed, not string-matched: Bash runs in Docker, Python runs against unit tests, and tool calls go through BFCL's AST checker.

• Tool calling is the big gain. The base model's tool-tag embeddings are untrained, so it barely emits its own <tool_call> format, and when asked for tools it mostly answers in Markdown JSON. Fixing that, plus the tool data, takes it from 52.7% to 86.2%.
• Bash and Python improve without being traded away. NL2SH-ALFA gains 26 commands and Python gains 38 problems, both significant. MultiPL-E sh (writing whole shell programs) is unchanged.
• InterCode needs a caveat. 127 of its 200 tasks overlap NL2SH-ALFA's test split, and the whole gain sits there. On the 73 independent tasks it is flat.

Compared with other small tool-calling models

Every model was run through the same harness in bf16, using its own chat template and native tool-call format.

Notes: * = the gap to mini-agent-v2 is statistically significant (both models answered the same questions; paired exact McNemar test, p < 0.05), so it is unlikely to be luck. No * means the two are effectively tied. ** = that model was not tested on this benchmark (not a score of zero).

---

Files

The weights are split across two repos: F-A-I-L/mini-agent-v2 holds the bf16 safetensors for transformers / vLLM, and F-A-I-L/mini-agent-v2-GGUF holds the GGUF builds for llama.cpp and GGUF apps.

GGUF metadata: architecture qwen2, 1,543,714,304 params, context 32768, EOS <|im_end|>, with the stock Qwen2.5 chat template (tool support intact). All quants were made from the bf16 GGUF with llama-quantize. Plan on the file size plus about 1 GB of memory.

Scores above are for the bf16 weights. Fewer bits per weight means a smaller file and lower quality. Every GGUF was checked to produce a correct tool call, but the quants were not re-benchmarked, so expect somewhat lower scores than these, more so the smaller the file.

---

How to prompt it

Tool calling: pass tools through the chat template's tools= field (or the OpenAI tools field on a server). No system prompt is needed; the template inserts the tool list. It replies with one or more <tool_call>{"name": ..., "arguments": {...}}</tool_call> blocks, or with plain text when no tool fits.

Bash

You are a Bash assistant. Translate the request into a single Bash command.

For a full script: You are a Bash assistant. Write a complete Bash script for the request.

Python

You are a Python coding assistant. Write correct code. Respond with a single code block.

The benchmark numbers on this card were measured with greedy decoding (temperature 0).

Usage

llama.cpp

# downloads the chosen quant from the GGUF repo on first run
# (QUANT = Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0 or bf16)
llama-server -hf F-A-I-L/mini-agent-v2-GGUF:QUANT --jinja -c 4096 -ngl 99
# or, with a file you already downloaded
llama-server -m mini-agent-v2-QUANT.gguf --jinja -c 4096 -ngl 99

--jinja is required for tool calling. Without it, llama.cpp ignores the template's tool support.

curl localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "messages": [{"role": "user", "content": "What is the weather in Berlin in celsius?"}],
  "tools": [{"type": "function", "function": {"name": "get_weather",
    "parameters": {"type": "object", "properties": {
      "city": {"type": "string"},
      "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}},
    "required": ["city"]}}}]}'
# -> tool_calls: get_weather({"city": "Berlin", "unit": "celsius"})

Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "F-A-I-L/mini-agent-v2"
tok = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16, device_map="auto")

tools = [{"type": "function", "function": {"name": "get_weather",
          "parameters": {"type": "object", "properties": {"city": {"type": "string"}},
                         "required": ["city"]}}}]
msgs = [{"role": "user", "content": "Weather in Paris?"}]
enc = tok.apply_chat_template(msgs, tools=tools, add_generation_prompt=True,
                              return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**enc, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))
# <tool_call>
# {"name": "get_weather", "arguments": {"city": "Paris"}}
# </tool_call><|im_end|>

Any GGUF app

LM Studio, Jan and KoboldCpp load the .gguf directly. To use tools, turn on the app's tool/function-calling support.

---

🖥️ Built on one 8 GB laptop GPU, looking for a bigger one

Every run behind this model was trained and evaluated on a single RTX 4060 (8 GB). That is why it's 1.5B, trained on a 4-bit base with 2k-token examples. The same weights already score significantly higher on Bash and Python when run in full bf16 instead of 4-bit.

If your team has an NVIDIA DGX Spark or an AMD Ryzen AI Max+ ("Strix Halo") AI dev kit to spare, it would go straight into full-precision training, longer context, 3B–7B bases and verified-reward RL, all released openly.

Get in touch via Discussions. No obligation; the model stays free either way. Thank you!

---

Evaluation

How this was measured

All results come from a run on 2026-10-04 (external models 2026-10-05), scored with each benchmark's own scorer. Where a task can be executed, it was executed.

• Decoding: greedy, batch size 8, 4096-token eval context, bf16 weights, the same harness for every model.
• Tool calling: BFCL non-live (simple, multiple, parallel, parallel-multiple) and live, in FC mode. Tools go through each model's tools= chat template, the answer is decoded from its native tool-call output, and it is checked with a reimplementation of BFCL's official AST checker. This is this project's harness, not the public leaderboard's prompt mode, so the numbers are not directly comparable with the leaderboard.
• Bash: NL2SH-ALFA test split, executed through InterCode-ALFA in Docker and scored on filesystem/output equivalence. InterCode-Bash runs in its own images. MultiPL-E sh runs unit tests in a network-less container.
• Python: EvalPlus HumanEval+ and MBPP+ (base + hardened "plus" tests), executed.
• Significance: paired exact McNemar test on per-question outcomes. Same problems, same prompts, same decoding.

Tool calling, in detail

The remaining errors are mostly argument values, not tool choice:

• Optional parameters left out or added when the user didn't ask for them
• User wording not mapped onto a schema enum value
• Date arithmetic ("next Tuesday")
• Very long tool lists

---

Development history

mini-agent-v2 is run b15 of a series:

• b1–b9 trained BFCL's prompt format ([f(a=1)]). It scored well on the benchmark, but no inference server parses it.
• From b10, every run is native-FC only. b10 also brought the tool-tag fix: in the base model, the <tool_call> / </tool_call> embedding rows are byte-identical untrained vectors with tied embeddings. LoRA doesn't touch embedding rows, so both were re-initialised and trained as extra tokens.
• b11–b15: each run fixed one failure class found by row-level error analysis of the previous run.
  • b12 reverted a rule that taught dropping optional parameters and removed rows with invented years.
  • b14 added targeted slices.
  • b15 removed the b14 slice that caused its non-live regression.
• b16 added longer context and a "fill discipline" filter. It came out null against b15, so b15 is the release.

The chart shows each run's 4-bit evaluation, which is why b15 reads 85.7% / 65.8% there and 86.2% / 65.7% in bf16 above. Neighbouring runs are often within noise: b15 vs b14 was p < 0.001 on non-live and p = 0.128 on live.

---

Training recipe

Data: 24,745 rows after the length filter: tool 49%, Python 28%, Bash 23%.

• Tool (12,275, all native FC, rendered through the real chat template):
  • xLAM calls, weighted toward parallel and multi-call shapes
  • Toucan MCP calls from real servers
  • 1,000 ToolACE rows
  • 1,000 full glaive loops (call → tool result → answer)
  • 500 "irrelevant" rows, where the tools are listed but the right answer is plain text
  • Slices for array arguments, enum mapping, date phrasing and a generic shell tool
  • No trained call contains a year the user didn't state.
• Python (7,000): LeetCode problems and stub completions, plus KodCode rows (CC-BY-NC-4.0, see License) whose answers were re-executed against their unit tests.
• Bash (5,670): NL2SH-ALFA train, bash-instruct-III one-liners and scripts, and linux-command-dataset.
• Leak protection: training prompts that match BFCL, NL2SH-ALFA test, HumanEval or MBPP+ were removed.

---

Known limitations

• About 1 in 3 real-user tool questions is wrong. See the error classes above.
• Not a multi-step agent. Training is mostly single-call, with 1,000 two-hop loops. On a Docker agent probe it completes 7/54 tasks, and 8/45 on held-out task families.
• Bash output is a single command or a short script, unverified. Sandbox it.
• The quants were not re-benchmarked. Expect lower scores than the bf16 numbers, more so the smaller the file.
• English only. No safety tuning beyond the base model.

Intended use and limits

Good for: local and offline tool-calling back-ends behind any OpenAI-compatible server, shell one-liner translation, small Python functions, and edge deployment.

Not for: production automation without validation, long autonomous agent loops, or anything safety-critical. Never execute generated shell commands unreviewed.

---

Citation

@misc{mini_agent_v2,
  title  = {mini-agent-v2: a 1.5B native tool-calling, Bash and Python fine-tune of Qwen2.5-Coder-1.5B-Instruct},
  author = {Frost2o24},
  year   = {2026},
  url    = {https://huggingface.co/F-A-I-L/mini-agent-v2},
  note   = {Merged bf16 weights and GGUF builds; evaluated on BFCL (FC mode), NL2SH-ALFA, InterCode-Bash, MultiPL-E sh and EvalPlus}
}

License

The model weights are released under Apache-2.0, inherited from Qwen2.5-Coder-1.5B-Instruct.

⚠️ Training-data notice: about 5,100 of the 24,745 training rows (the KodCode Python slices: 4,300 general + 800 odd-function-name rows, via F-A-I-L/kodcode-verified-python-235k) come from KodCode-V1, which is licensed CC-BY-NC-4.0 (non-commercial). All other training sources are Apache-2.0, MIT or CC-BY-4.0. Whether a dataset's license carries over to a model trained on it is legally unsettled. If you plan to use this model commercially, review that dataset's terms and decide for yourself; this notice is not legal advice.

Training data licenses are those of the upstream sources listed in the metadata. xLAM (CC-BY-4.0) is credited to Salesforce Research.