Qwen3.5-4B-Salieri

Salieri is a 4B coding-agent model fine-tuned from Jitheswar/Qwen3.5-4B-Amadeus (an abliterated Qwen3.5-4B). It is trained to work as an autonomous software-engineering agent: explore a repository with tools, find the cause of an issue, edit code, run targeted tests and submit a patch. It keeps Qwen3.5's hybrid thinking, native tool-calling format, 262K context and vision tower (untouched by training).

v2 (current main): a second round of step-level DPO on top of v1. v1 stays available at revision v1. v2 was accepted only under a rule written down before its held-out results were seen: dev win, held-out + OOD at least v1's, and no code-benchmark drop over 1 point.

Benchmarks

All numbers are our own runs with identical settings for every model (thinking mode, temperature 0.6, top-p 0.95, top-k 20, vLLM 0.32). Agentic numbers pool 2 seeds.

v2 vs v1 differences on the agentic splits are a few tasks out of 60-252 runs and go both ways (v2 is better on held-out repos, worse on post-release OOD and bash-only): within seed noise on each split. If you prefer the model that was checked on the lockbox, use revision v1. The clearest effects of the overall recipe (v1 and v2 alike) are LiveCodeBench and IFEval gains over the base and far fewer action loops.

How the agentic benchmark works. SWE-style tasks are real GitHub issues with hidden tests, run in a docker-free sandbox with 6 generic tools (run_command, read_file, edit_file, write_file, get_status, submit_patch), at most 60 tool calls and 20 minutes per task, and a short generic system prompt. A task counts as resolved only if the hidden tests pass.

• held-out repos (holdout_sound, ood_sound): repositories never seen in training; ood_sound contains bug fixes merged after Qwen3.5's release.
• lockbox (ext_final, 67 tasks from 15+ further repositories): never used for any decision; evaluated a single time (2 seeds), for v1 only, after its selection was frozen; it was not reused for v2.

No benchmaxing. Training never used any benchmark repository: every trajectory from a repository that appears in any evaluation set was removed (522+ trajectories dropped). Checkpoints and recipes were chosen on separate dev splits only, the system prompt used for evaluation is generic, and the harness/prompt-optimization loop (Harness2) ran on training-pool tasks only.

Training

Two rounds of step-level DPO, both LoRA r64 (Unsloth bf16), beta 0.1 + 0.2 chosen NLL (RPO), 2 epochs, lr 1e-5. Training used training-pool tasks only (no dev / held-out / OOD / lockbox repositories).

1. v1, against action loops. Fine-tuned small agents get stuck re-issuing the exact tool call they just made. From agent rollouts on the training pool, 558 states where a model began repeating itself were collected; Qwen3.8-27B wrote the next step in the same context (4 samples, keeping a clean call that differs from the repeat), giving 448 pairs: chosen = teacher step, rejected = the repeat. Reference = the base. Held-out pair accuracy 0.95; in-budget exact-repeat rate fell from 6.3% to 0.4% on dev.
2. v2, iterated from v1. 246 new pairs, reference = v1: 234 loop states from rollouts of a different (full fine-tune) checkpoint, plus 12 "commit to an edit" states where the agent had explored for 30+ tool calls without editing and the teacher chose to edit in at least 2 of 4 samples. Held-out pair accuracy 0.96.

Usage

vllm serve Jitheswar/Qwen3.5-4B-Salieri --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

Recommended sampling: temperature 0.6, top_p 0.95, top_k 20 (thinking). Works with any OpenAI-compatible agent harness that uses native tool calls. GGUF quantizations (BF16, Q8_0, Q6_K, Q4_K_M + mmproj) are included for llama.cpp / LM Studio / Ollama.

Limitations

A 4B model: it solves a minority of real repository issues, can loop on hard tasks, and was evaluated mainly in one docker-free harness (plus the bash-only tool-set check above). Abliterated base: it does not refuse much; deploy with your own safeguards.

Credits

• Base: Qwen team (Qwen3.5-4B, Apache-2.0), abliteration: Qwen3.5-4B-Amadeus.
• Teacher: Qwen3.8-27B (Apache-2.0).
• Data: nvidia/Open-SWE-Traces (CC-BY-4.0; tasks from Scale-SWE and SWE-rebench-V2) — Ahmad et al., Open-SWE-Traces, arXiv:2606.16038.
• Methods: iterated DPO (Rafailov et al. 2023) with an NLL term (RPO, Pang et al. 2024), applied at the step level to agent states. Evaluation harness tuned with Harness2 (google-research/harness2, arXiv:2610.14224) on training-pool tasks only. Training with Unsloth.