MiniCPM-V-4.7-1B-Agentic-FwS

A tool-calling fine-tune of openbmb/MiniCPM-V-4.7-1B, trained on the tool_calling split of nvidia/Nemotron-SFT-Agentic-v4 (OpenMDW-1.1). The model thinks inside <think>...</think>, then emits one or more <tool_call> blocks.

There have been two supervised training rounds and one reinforcement-learning run on top of them. Each was trained as LoRA adapters and merged into the fp16 weights. Tensor names and shapes are identical to the base model, so any loader for MiniCPM-V-4.7-1B loads this repo too.

Contents

• Results
• Where the model still fails
• Usage
• Evaluation protocol
• Round 1 method
• Round 2 method
• Experiments that did not help
• Training compute and speed
• Related models
• Artifacts
• Limitations
• References

Results

The held-out val set is 1000 Nemotron-SFT-Agentic-v4 tool_calling rows. All are dataset rows at index 20000 or above, and no training round used them. Scoring is on the first tool-call turn with greedy decoding.

Round 2 compared with round 1 on the same 1000 rows (paired):

All four metrics are higher than round 1.

v9 (main) compared with v6 (paired, 1000 rows): exact +0.005, standard error 0.0102 (55 rows newly right, 50 newly wrong); name -0.001; args equal; has_call 1.000 vs 0.999. This is a tie. It is below the 2-standard-error threshold used to accept earlier runs, and name is 0.001 lower. It is main because the owner chose it, not because it passed that test. Use revision f7c6ea0 for v6.

Reference run: Claude Haiku 5.5 on the same 1000 rows and the same four metrics. Paired difference, v9 (main) minus Haiku: exact +0.080 (standard error 0.0141; 142 rows only v9 right, 62 only Haiku right), args +0.068 (0.0147), name +0.107 (0.0161). v6 minus Haiku is similar (exact +0.075, standard error 0.0144). Read this with care:

• The val rows are in Nemotron's own style, and these models trained on that style. The numbers show agreement with the dataset's answers, not general tool-use ability.
• Haiku was run as a Claude Code subagent (25 prompts per agent, one reply per prompt), under that agent's own system prompt, not through a raw API call.
• 105 of its replies were plain text, while every gold answer is a tool call. On prompts that chain several steps it often sent only the first call.
• A first pass used larger batches and the agents lost context and guessed, so those batches were discarded and rerun in groups of 5 rows. A few kept rows were still flagged by the agents as inferred or reconstructed and were not re-verified.

v6 compared with v5 (paired):

Older 128-row benchmark

Round 1 was first measured on a 128-row held-out set. On that set round 1 scored 0.422 exact; on the 1000-row val set it scores 0.314. The small set overstated accuracy, so its numbers cannot be compared with the table above.

Where the model still fails

Error types on the 1000 val rows:

Exact match and args match broken down by the gold call, shown as exact / args:

Why the reachable score is well below 1.0:

• Gold values not in the prompt. Of the 693 val rows where gold values can be checked (values of 3+ characters), 416 (60%) have at least one gold value that does not appear verbatim in the prompt. 49% of the 1099 checked values are ungrounded. Mostly free-text queries, locations and placeholder keys. Round 2 scores 0.52 exact on the 277 rows whose values all appear in the prompt and 0.10 on the 416 rows with an ungrounded value.
• Placeholder API keys are not a learnable convention. When a called tool has an optional API-key parameter, gold fills it on only 17-20% of calls in the pool, the training rows and val, and leaves it out on the other 80%. When it fills it, the value is demo for 22-27% of those calls (others: test, empty, dummy, default, ...). Of 103 training tools with 5+ such calls, none always includes the key and 79% are mixed. Always adding demo would drop val args from 440 to 293 rows right.
• Text before the call. 69 gold targets put free text before the call, and exact match is always 0 on those rows. Round 2 almost never writes such text (1 of the 69 rows). With the text removed from gold and output, 25 of the 69 calls match, so exact on the call alone is about 0.434.
• Schema quirk. One tool's parameter is literally named code: (with the colon). Round 2 wrote code:" or code in 6 rows, losing on the key name although the code was right.

What the argument errors are (392 rows with the right name and wrong arguments): the output omits a gold key in 157, has the right keys but a wrong value in 193, adds a key in 36, and differs only in formatting in 6. Of 202 omitted keys, 195 are optional parameters in the schema, 54% of those with gold values that appear nowhere in the prompt.

Parallel calls. On the 47 val rows with 2+ gold calls, round 2 produced fewer calls than gold in 42 and more in none, and got the function names right in only 4.

Long calls, calls with many parameters and parallel calls remain the weak spot.

Usage

Requires transformers>=5.18.0.

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

repo = "ewin-reg/MiniCPM-V-4.7-1B-Agentic-FwS"   # revision="ab2a7d9" for round 1, "f7c6ea0" for v6
model = AutoModelForImageTextToText.from_pretrained(repo, dtype=torch.float16, device_map="cuda").eval()
tok = AutoProcessor.from_pretrained(repo).tokenizer

tools = [{"type": "function", "function": {
    "name": "get_weather", "description": "Current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]
messages = [{"role": "user", "content": "What's the weather in Jakarta?"}]

prompt = tok.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True, enable_thinking=True)
ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids.cuda()
im_end = tok.convert_tokens_to_ids("<|im_end|>")
out = model.generate(ids, max_new_tokens=768, do_sample=False, eos_token_id=im_end)
print(tok.decode(out[0, ids.shape[1]:]))

Output format, after the <think> block:

<tool_call>
<function=predict_gender>
<parameter=names>
Niko West, Alessia Romano, Boris Petrov
</parameter>
</function>
</tool_call>

Parallel calls appear as several <function=...> blocks. Parameter values that are not plain strings are written as JSON. A parser that matches the scoring code:

import re, json
def parse_calls(text):
    text = text.split("</think>", 1)[-1]
    calls = []
    for name, body in re.findall(r"<function=([^>\n]+)>(.*?)</function>", text, re.S):
        params = {}
        for k, v in re.findall(r"<parameter=([^>\n]+)>(.*?)</parameter>", body, re.S):
            v = v.strip()
            try: v = json.loads(v)
            except Exception: pass
            params[k.strip()] = v
        calls.append((name.strip(), params))
    return calls

Training and evaluation both used greedy decoding with thinking on. Sampling has not been evaluated.

Evaluation protocol

Metrics (the output is the text after </think>):

• exact: string equal to the gold turn.
• args: parsed (name, parameters) list equal to the gold list, order-free, with JSON-decoded values.
• name: sorted function names equal.
• has_call: <tool_call> present.

Decision rule. A candidate was kept only if its exact-match gain over the incumbent, paired per row, was at least 2 paired standard errors: se = sqrt(w + l - (w - l)^2 / n) / n, where w is rows won, l rows lost and n the number of rows. The final merge also required that no metric fall below round 1.

Round 1 method

Kaggle, 2x T4.

1. Finetuning with Sampling (FwS) (arXiv 2610.02140, official code aakaran/finetuning-with-sampling).
   • The expert reasoning of each training tool-call turn was rewritten by projection sampling. MiniCPM-V-4.7-1B itself served as both base and proposal model.
   • Settings: block 32, 10 MCMC steps per block, max 1856 tokens, temperature 0.6, repetition penalty 1.05. A rewrite was accepted when its mean base log-likelihood, given the task alone, was higher.
   • The expert tool call was kept verbatim.
   • 443 rows were boosted. With the boosted reasoning, greedy decoding still reached the expert call in 15.6% of rows, vs 16.3% with the original reasoning.
2. ASFT (arXiv 2509.23753, official code zhuchichi56/ASFT).
   • Loss: DFT + 0.05 x KL(base || policy), on 438 boosted examples with closed reasoning.
   • Recipe: loss on the response only, thinking on, max length 2048, batch 16, 2 epochs, AdamW, cosine, 10 warmup steps, clip 1.0. 54 / 54 steps.
3. DPO + NLL (Iterative RPO, arXiv 2404.19733, one iteration).
   • Rejected response: the SFT model's own greedy output, when its call was wrong. Chosen response: the SFT target (boosted reasoning + expert call).
   • 258 pairs; 173 of the 438 prompts were already right and gave no pair.
   • Settings: reference = SFT model, beta 0.1, alpha 1.0, LoRA r32, lr 1e-5, batch 16, 2 epochs. 32 / 32 steps.

Deviations from the papers (hardware limits):

• LoRA merged into the weights instead of full fine-tuning.
• SFT lr 1e-4 instead of 5e-5; preference lr 1e-5 instead of 7e-7.
• One greedy rejected sample per prompt instead of 30.
• Sampling rewrote only the reasoning of tool-call turns. The paper rewrites whole math and chemistry answers.

The cleaned round 1 data (281 SFT rows, 400 preference pairs, 360 distinct prompts) is public at ewin-reg/Nemotron-Agentic-ToolCalling-FwS-MiniCPM-V-4.7-clean.

Round 2 method

Plain LoRA SFT on more in-distribution gold data, using the recipe of arXiv 2609.17848, App. A.2, unchanged. One Colab T4, fp16.

Data fixes

They came from analysing why the first attempts lost tool calls.

The first plain SFT run (2000 rows, 64 steps) raised exact to 0.342, but has_call fell from 0.997 to 0.946.

• 50 of its 54 no-call rows never closed <think> within 768 tokens, and 35 of those were loops.
• 125 of the 2000 gold targets themselves contain a 10-gram repeated 3 or more times.
• Gold reasoning averages 188 tokens; round 1's own reasoning averages about 121.

Three fixes (FIX=2), each from the literature:

1. Loop filter. Gold targets with any 10-gram repeated >= 3 times are dropped (arXiv 2512.12895).
2. Self-replay. Correct tool calls sampled from earlier models on training prompts are mixed in, to keep their behaviour (arXiv 2402.13669, arXiv 2605.26097):
   • round 1: 4 samples per prompt at T 0.8, 360 prompts;
   • the STaR model: 10 samples per prompt.
3. Length cap. Gold targets longer than the 95th percentile of the replay targets (194 tokens) are dropped.

The loop filter plus replay alone (v2) gave exact 0.373 but has_call 0.972. Adding the length cap (v3) brought has_call to 0.996.

Continued runs

Every run started from the previous best adapter, never from scratch. Each of v6, v7 and v8 used examples the earlier runs had never seen.

The source split has 549,707 tool_calling rows. Rows 0-19999 are the training pool (18,547 examples after turn extraction and de-duplication); val and bench were taken from rows 20000 and up (bench rows 20000-21418, val rows 21419-22824). Rows 22825 and up (526,882) were never used until v8.

v8 (targeted selection from the unused rows) tested whether choosing rows helps where plain extra data had stopped helping (v7). Changes against the v6 recipe: only examples whose gold values all appear in the prompt (60% of rows with checkable values have a gold value that does not, which can teach the model to invent values); about 17% multi-call turns (479 of 2795 gold examples) instead of the natural share; the length cap exempted multi-call turns; examples whose prompt equals a val or bench prompt were dropped. Result: val exact 0.405 against 0.409 for v6, a tie within noise and below the keep threshold (a gain of 2 standard errors), so v6 stayed the published model until v9.

The v6 adapter was merged into round 1 in fp32, cast to fp16, and pushed only after a check that every tensor name and shape matched round 1's model.safetensors. The v9 weights were produced the same way from the ar3_flash adapter.

Experiments that did not help

All of these used the same base and the same evaluation.

RL gave no gain at this model size and budget. Each step needs many 768-token rollouts and took minutes. GRPO, JOLT+ and FlashREINFORCE all stayed inside noise. FlashREINFORCE (40 updates, about 3 h on a T4) was stable (the drift check rejected 0 of 64 trajectories at every update, tool-call rate stayed at 1.0) but did not move val exact beyond noise; more updates or a tuned LoRA learning rate were not tried.

Ideas considered but not run:

• Converted ToolACE, xLAM-60k and Nemotron-RL Pivot-v1 data. A Pivot-v1 conversion kept 907 of 9620 rows; those targets have empty reasoning.
• A separate weight on call tokens (BalanceSFT, a two-term loss; untested here).
• Gold-conditioned self-distillation (untested here).

Training compute and speed

Related models

Artifacts

In the public data and checkpoint repo ewin-reg/nemotron-sft-agentic-v4-fws-minicpmv47:

The ar3_flash/ckpt/adapter LoRA, applied to revision ab2a7d9, reproduces main up to fp16 rounding. The ar2_sft7/ckpt/adapter LoRA, applied to ab2a7d9, reproduces revision f7c6ea0 (v6).

Limitations

• Only the first tool-call turn of Nemotron-style prompts was evaluated. Multi-turn agent loops (feeding tool results back), other tool-call formats, other languages and image inputs were not tested.
• Vision and general chat quality were not measured after fine-tuning and may have regressed.
• Exact match is 0.04-0.10 on calls with 3+ parameters or parallel calls. Validate arguments before executing them.
• The model sometimes invents argument values (IDs, keys) the user did not give. Confirm before calling tools with side effects.
• Label noise in the val set limits what exact match can show; args and name are more informative. All scores come from Nemotron's own val rows; no outside tool-calling benchmark (for example BFCL) was run, so gains beyond Nemotron-style prompts are unmeasured. The Haiku 5.5 reference row is a single run on the same in-distribution rows, so it does not show which model is better at tool use in general.

References

• Finetuning with Sampling: arXiv 2610.02140
• ASFT: arXiv 2509.23753
• Iterative RPO (DPO + NLL): arXiv 2404.19733
• LoRA SFT recipe: arXiv 2609.17848
• Repetition filtering: arXiv 2512.12895
• Self-replay against forgetting: arXiv 2402.13669, arXiv 2605.26097
• STaR: arXiv 2203.14465
• STAR (tool-call RL reward): arXiv 2602.03022
• JOLT: arXiv 2610.10447
• WiSE-FT: arXiv 2109.01903
• Data: nvidia/Nemotron-SFT-Agentic-v4