LFM3 1.7B Claude Code

hi, I’ve fine-tuned the LiquidAI/LFM3-1.7B model over several rounds using the mondk/Ultra-Trio-Unfiltered dataset:

• Round 1: Tool usage
• Round 2: Improving benchmark scores
• Round 3: RL (the model generates its own questions and answers)

safetensors: mondk/LFM3-1.7B-Claude-Code-Safetensors

---

1. Results

Method

• BFCL: function calls matched against gold.
• LCB: final code run against all gold tests; original outputs kept where the model finished its answer.
• TerminalBench and SWE: correct only if resolved (all verifier tests pass).
• Both models: f16, temperature: 0, thinking: default. Max tokens: 1200 (BFCL), 2048 (LCB), 2560 (TerminalBench and SWE). Max tool calls: 9.

Limitations

• Sample size is small: 30 BFCL questions, 12 LCB questions, 4 TerminalBench questions and 4 SWE questions. Results may differ slightly from other evaluations.
• Some BFCL answers that use JSON or prose instead of the required [func(args)] format were counted as correct if the call content was right.
• TerminalBench and SWE come from noodcon/TerminalBench-DockerLess and noodcon/SWE-DockerLess, Docker-free versions of the official benchmarks. Docker was too complex to set up here, so these may not match the official benchmarks.

2. Process metrics

ty!