LFM2.5-230M-tools GGUF

The LFM2.5-230M tool-calling fine-tune in GGUF format, for llama.cpp-style runtimes. The model was supervised-fine-tuned with LoRA so it emits tool calls for agentic workflows. The full BF16 model and its validation table live at iromu/LFM2.5-230M-tools.

Base model

This model is the GGUF conversion of:

iromu/LFM2.5-230M-tools

— itself a LoRA fine-tune of LiquidAI/LFM2.5-230M-Base.

Tool-call format

The calls in these files are not OpenAI-shaped. The template renders a call as a bracketed list between two markers, with argument values in a Python-like form:

<|tool_call_start|>[search_files(path="src/", query="load"), read_file(path="src/app.py")]<|tool_call_end|>

A runtime consuming these files has to know the format. A parser that looks for an OpenAI tool_calls array finds nothing on every row, which reads like a model that never learned to call tools.

GGUF files

The model is provided in GGUF format at the following precisions:

No imatrix-guided (UD-*) family ships with this model. Those quants are built from a per-tensor importance recipe, and none exists for this family — guessing one is not an option, so the set above is the whole of it.

Intended use

• Structured tool / function calling in a very small footprint
• On-device and edge deployment with llama.cpp-style runtimes

It is not a drop-in for a runtime that assumes the OpenAI tool-call envelope; see the format section.

Usage

Load a GGUF quant with llama.cpp:

llama-cli -hf iromu/LFM2.5-230M-tools-GGUF:Q4_K_M

Or with the llama.cpp Python bindings:

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_path="iromu/LFM2.5-230M-tools-GGUF",
    filename="LFM2.5-230M-tools-Q8_0.gguf",
    n_gpu_layers=99,
)

Validation

Every fine-tune row above is the GGUF file itself, served by a llama-server and scored through the same scorer and decoding path, at the same 384-token cap, on the same 274-row split (272 gradable). The base row is the upstream base, which the pipeline scores through transformers; it produced no parseable call on any row, so every delta against it is the fine-tune's absolute number rather than a gain over something the base could do.

The rows do not line up as a quantization ladder, and that is the honest result, not a mistake. Greedy decoding on this model sits on a knife-edge: between the BF16 file and any one quant, 54 to 92 of the 272 rows flip their emit-or-not decision, in both directions, and only 131 rows are decided the same way by all four files. That is why Q5_K_M emits more than the BF16 file (36.4% vs 22.4%) while Q4_K_M emits fewer (12.9%) — quantization perturbs the logits enough to move borderline rows either way, and each headline count is the net of many such flips, not a monotonic function of bits-per-weight. Read a row as what that file does under llama.cpp, and compare files by the rows that flip rather than by the headline number alone.