Qwen3.5-0.8B-ita-agent-MLX-8bit

A small Italian agent for MCP tools and the terminal: Qwen3.5-0.8B fine-tuned with LoRA on verified synthetic conversations, fused and quantized to 8-bit MLX. It reads and writes files, manages notes and calendar events, computes dates and VAT with tools, fetches web pages, asks before deleting or changing data, and answers plain chat in Italian. It is not a coding model.

Everything was done at home on two Mac mini M4 with 16 GB: data generation, training and evaluation. No rented GPU, no closed model output in the training data.

Results

All numbers are greedy decoding, measured on this fused 8-bit model unless stated otherwise. Tasks run in deterministic simulated environments served as real MCP servers (files, notes, calendar, web, terminal, Italian date and tax tools); a task passes only if the environment state and the answer pass their checks.

The dev split has 10 categories of 8 to 10 tasks: single call, chain, multi-turn, parallel calls, no tool needed, clarify, recovery after a tool error, summarize, choice, safety. Fusion quantizes the weights again: the adapter before fusion scored 54/88 on dev and 207/296 on fresh5, and the fused model passes the same dev tasks and differs from it on 9 fresh5 tasks, within noise. Safety (ask before deleting or overwriting): 8/10 on dev, 21/26 on fresh5.

The frozen test split is the main number: 112 tasks written before training and read once, after every choice was made, 45 of them on two environments the model never saw in training (TLS certificates and git). Dev guided the harness and the training data, so it overstates this model by about 18 points.

Against the stock 0.8B run plainly, as most people run it, the gain holds: with its harness this model wins 32 test tasks and loses 9 (p < 0.001), and run plainly it still wins 24 and loses 9 (p = 0.014). It passes 49 test tasks, as many as the stock Qwen3.5-2B 8-bit in the same harness (48), but fewer than the stock 2B run plainly (58), which does better without the harness. It asks before a destructive action in 9 of 11 test safety tasks, against 1 of 11 for the plain stock 0.8B and 3 of 11 for the stock 0.8B in the harness.

The agent harness

The best numbers come with a thin harness around the model, part of the training setup:

• each user turn is first classified by the model itself as answer, tools, ask or confirm, and the turn then runs with that intent;
• in an ask or confirm turn, tools that delete or change data are hidden and refused;
• file editing uses replace_in_file and append_to_file instead of whole-file rewrites, and a JSON or Python file read or written gets a syntax verdict;
• the context lists the calendar from today for the next seven days.

Without the harness the model still calls tools in the trained format, with the lower score in the table. Safety is where the harness matters most: without it the model asked before a destructive action in 4 of 10 dev safety tasks, against 8 of 10 with it.

Training

• Base: Qwen/Qwen3.5-0.8B, 8-bit MLX (group size 64), the model's own chat template and the qwen3_coder tool-call format.
• LoRA rank 16, scale 20, all layers, learning rate 2e-5 with 30 warmup steps, one epoch, sequences up to 3,072 tokens; 10.8M trainable parameters; 3,322 steps on two Mac mini M4 16 GB as a ring (gradients averaged every step), with MLX.
• Data: 2,567 conversations and 3,576 turn-classification examples, generated by Qwen3.5-4B (8-bit, Apache-2.0) acting in 61 task families of simulated Italian worlds, with tool faults injected in part of the tasks. A conversation is kept only if the environment checks pass. Evaluation tasks and the test-only environments never appear in the data; a contamination check of the test requests against the data found no copies.
• Final validation loss 0.128.

Use

from mlx_lm import load, generate

model, tokenizer = load("fabriziosalmi/Qwen3.5-0.8B-ita-agent-MLX-8bit")
tools = [{
    "type": "function",
    "function": {
        "name": "list_directory",
        "description": "Elenca file e cartelle in un percorso",
        "parameters": {
            "type": "object",
            "properties": {"path": {"type": "string", "description": "Percorso della cartella"}},
            "required": ["path"],
        },
    },
}]
messages = [{"role": "user", "content": "Che file ci sono in /home/utente/documenti?"}]
prompt = tokenizer.apply_chat_template(
    messages, tools=tools, add_generation_prompt=True, enable_thinking=False
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

tools holds the JSON schemas of the MCP tools you connect. A tool call comes back in the qwen3_coder format of the base model; your client runs it and sends the result back as a tool message. Keep enable_thinking=False: the model was trained and evaluated without thinking.

Limitations

• Measured on simulated environments, not yet on real MCP servers.
• The 2B is stronger: the fused 2B trained on the same data passes 66 test tasks against 49 for this model (23 against 6 where they differ, p = 0.002). Use the 2B when memory allows.
• Weakest areas on the test: clarifying an ambiguous request (1 of 10), chains of dependent calls (2 of 10) and recovery after a tool error (4 of 10).
• Without the harness it asks before a destructive action less than half of the time: keep destructive tools behind a confirmation in your client.
• Italian only. Thinking mode was off in training and evaluation.

License

Apache-2.0, as the base model and the teacher.