LFM3-1.7B-HelpAgent
A fine-tune of LFM3-1.7B that answers questions about llama-swap. It looks up the answer with llama-swap's documentation and config tools, and helps write config files.
It runs on llama-server, or entirely in the browser with Transformers.js and
WebGPU. See Run it in the browser.
It is built on Liquid AI's LFM3-1.7B and is not made, affiliated with or endorsed by Liquid AI. See License.
Files
| File | Size | Use |
|---|---|---|
LFM3-1.7B-HelpAgent-Q4_K_M.gguf |
1.1 GB | llama-server. This is the file the scores below measure. |
model.safetensors |
3.5 GB | bf16 weights for transformers or SGLang, or for making other quantizations. |
onnx/model_q4f16.onnx |
1.4 GB | Browsers with the shader-f16 WebGPU feature. |
onnx/model_q4.onnx |
1.6 GB | Browsers without shader-f16. |
Download the Q4_K_M GGUF, or from a terminal:
curl -LO https://huggingface.co/llama-swap/LFM3-1.7B-HelpAgent/resolve/main/LFM3-1.7B-HelpAgent-Q4_K_M.gguf
How it was evaluated
The Docs Agent eval in the llama-swap repository runs the real agent loop against real tools. Each question gets 3 attempts, and each attempt is graded by regex assertions. The training cases guided the work. The holdout cases were not trained on.
Q4_K_M on llama-server, with 60 seconds per attempt:
| Model | Training cases (134) | Holdout cases (51) |
|---|---|---|
| LFM3-1.7B, stock | 44.8% | 55.6% |
| LFM3-1.7B-HelpAgent | 95.5% | 96.1% |
An attempt took a median of 12 s and 32 to 34 s at the 95th percentile, on one RTX 3090 at concurrency 4. The stock model's low scores are mostly timeouts: 276 of its 555 attempts ran out of time, against 9 for this model.
The holdout score moved between 92.8% and 97.4% across runs with different repeat penalties and time limits, so treat small differences as noise.
Limitations
- It only knows the llama-swap documentation it can reach through its tools, at the version it was trained on. It is not a general assistant.
- It still invents config keys. An LLM judge flagged this in about 1 of 6 answers, which regex grading cannot see. Check generated config against the schema.
- About 1.6% of attempts did not finish in 60 seconds, because the model reasoned for too long or repeated tool calls.
- It has only been tested with llama-swap's system prompt and tools.
Usage
llama-server needs llama.cpp b11960 or later. These are the settings used for the
scores above:
llama-server -hf llama-swap/LFM3-1.7B-HelpAgent:Q4_K_M --jinja \
--ctx-size 131072 --parallel 4 --kv-unified \
--temp 0.1 --top-k 50 --repeat-penalty 1.1
Requests need llama-swap's
Docs Agent system prompt
and the tools from llama-swap's /api/mcp endpoint. Tool calls use LFM3's XML
format, which llama-server parses with --jinja.
With transformers, load the model with trust_remote_code=True. Read the included
Python files first.
Run it in the browser
The ONNX models run with no server, using Transformers.js 4.3.2 and WebGPU. The browser downloads the model from this repository once and caches it. This needs a helper, because Docs Agent prompts are long:
import {
AutoModelForCausalLM, AutoTokenizer, DynamicCache, Tensor, ones,
} from '@huggingface/transformers';
const id = 'llama-swap/LFM3-1.7B-HelpAgent';
const adapter = await navigator.gpu?.requestAdapter();
if (!adapter) throw new Error('WebGPU is not available');
const dtype = adapter.features.has('shader-f16') ? 'q4f16' : 'q4';
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await AutoModelForCausalLM.from_pretrained(id, { dtype, device: 'webgpu' });
// A long prompt in one pass runs out of memory, so read it in chunks of 256 tokens.
async function prefill(ids, size = 256) {
const cache = new DynamicCache();
const total = ids.dims[1];
for (let start = 0; start < total - 1; start += size) {
const end = Math.min(start + size, total - 1);
const output = await model({
input_ids: ids.slice(null, [start, end]),
attention_mask: ones([1, end]),
past_key_values: cache,
num_logits_to_keep: new Tensor('int64', [1n], []),
});
output.logits.dispose();
cache.update(Object.fromEntries(
Object.entries(output)
.filter(([name]) => name.startsWith('present'))
.map(([name, tensor]) => [
name.replace('present_conv', 'past_conv').replace('present', 'past_key_values'),
tensor,
]),
));
}
return cache;
}
// messages and tools come from your agent loop.
const prompt = tokenizer.apply_chat_template(messages, {
tools, add_generation_prompt: true, tokenize: false,
});
const inputs = tokenizer(prompt, { add_special_tokens: false, return_tensors: 'pt' });
const cache = await prefill(inputs.input_ids);
const output = await model.generate({
...inputs, past_key_values: cache, max_new_tokens: 1024,
});
await cache.dispose();
const reply = tokenizer.batch_decode(
output.slice(null, [inputs.input_ids.dims[1], null]),
{ skip_special_tokens: false },
)[0];
The reply holds raw <think> and <tool_call> text, so your agent loop has to parse
it, run the tool and send the result back. generate() reads the sampling settings
from generation_config.json.
Requirements and limits
- Choose the file by
shader-f16.q4f16loads without it, then fails on the first generation withGatherBlockQuantized requires f16 but the device does not. Chrome 158 on Linux did not offershader-f16on an RTX 3090, so check for it rather than assuming a large GPU has it. - Always prefill in chunks. In one pass,
q4ran a 24 GB GPU out of memory at 7,767 tokens, and Liquid AI's notes report wrong output fromq4f16at 8,192. With chunks, memory stays low:q4f16peaked at 3.2 GB of VRAM with a 14,170 token prompt. - Use Transformers.js 4.3.2. It is the only version tested.
Test results
On an RTX 3090, with seven real Docs Agent prompts of 3,622 to 14,170 tokens, and sampling off so runs can be compared:
q4f16 |
q4 |
|
|---|---|---|
| Runtime | Node with onnxruntime-node's WebGPU, not a browser |
Chromium 158 with WebGPU |
| Load from local disk | 5 s | 10 s |
| Prefill, 14,170 tokens | 6 s | 9 s |
| Generation | 31 to 48 tokens/s | 23 to 27 tokens/s |
| Prompts completed | 7 of 7 | 7 of 7 |
Every prompt rendered byte-identically to Python, with the same token count. Chunked
and single-pass logits matched closely: a mean difference of 0.02 on a range of about
30, with the same top 5 tokens, so chunks can change a word now and then but not the
behavior. q4 and q4f16 wrote the same reply on 4 of the 7 prompts, and differed
only in wording on the others.
Not tested:
q4f16inside a real browser, because no browser here offeredshader-f16. Test it on a machine that has the feature.- macOS, Windows, or any GPU other than an RTX 3090. Expect a smaller GPU to be slower.
- The Docs Agent eval. The scores above are for the GGUF, not the ONNX models.
Changes from a standard export
onnx/ was made with Liquid AI's
onnx-export at commit 26857c2, which
writes logits_to_keep as an input, where Transformers.js sends
num_logits_to_keep. These graphs take the latter, so lfm3-infer from that
project will not run them. The ONNX weights were quantized from the same
fine-tuned weights as the other files.
Training
- Base: LFM3-1.7B, revision
0dc4268b561499c648c7ba7c9466bf3d8d01517a - Method: LoRA, then merged. r 16, alpha 32, dropout 0.05, learning rate 2e-4 with cosine decay, batch 1 accumulating 8, 2 epochs, bf16
- Data: 417 conversations, 1,329 assistant turns. A stronger model,
Q3.8-27B-rad, ran the real agent loop against llama-swap's documentation, and each conversation was filtered: it had to finish in few steps, and every config key in its answer had to appear in a tool result. No eval case was trained on. - Hardware: one RTX 3090, 87 minutes, 9 GB of VRAM
The work is tracked in llama-swap #1302.
License
This model is a derivative of LFM3-1.7B and is distributed under the LFM Open License v1.0. The license allows free commercial use only for organizations with annual revenue under $10 million USD. Above that, see Liquid AI's license guide. Research use by anyone, and non-commercial use by qualified nonprofits, is covered.
Modified files are listed in NOTICE. model.safetensors, the GGUF and
the ONNX models contain modified weights. config.json and tokenizer_config.json
have additions for Transformers.js. All other files are unchanged from LFM3-1.7B,
except this model card.
The license grants no rights to Liquid AI's trademarks. "LFM3" appears in this model's name only to describe where it came from.
Citation
Please cite the base model:
@article{liquidAI2026LFM3,
author = {Liquid AI},
title = {Introducing LFM3: Every Modality for Specialized Intelligence},
journal = {Liquid AI Blog},
year = {2026},
note = {www.liquid.ai/blog/lfm3},
}