dale-4b (GGUF)

A full fine-tune of Qwen/Qwen3.5-4B that turns a natural-language request, in Spanish or English, into one bash command for Linux, answered as a small JSON object. It was trained with the same data as dale-0.8b. It is one of the two GPU models of dale (since version 1.1.0), an offline command-line tool that runs it through llama.cpp with Vulkan, and it works with llama-server in any other application, on CPU or GPU.

• Author: Manuel García Sanz
• Website: https://manuelgs.com/en/dale/
• Source code of dale: https://github.com/manuelgscom/dale
• Format: GGUF, Q4_K_M (imatrix) · 2,708.8 MB
• Test v2 accuracy: 87.0 % (410/471) with llama-server, like every model in the comparison; 86.6 % (408/471) through dale on the GPU with Vulkan
• In the dale CLI: used on the GPU when a dedicated GPU has at least 4 GiB of free VRAM. With 8 GiB or more it is the default answer (Enter) of the 4B/9B question. About 1.2 s per request and 3.2 GiB of VRAM (details)
• The dale family: four sizes of the same model, all in one collection

The dale CLI live: ask in plain English; it proposes the command, explains it, rates the risk and only runs it once you confirm. Real, unedited recording with dale-0.8b on a desktop CPU; with dale-4b on the GPU the flow is the same, with a more accurate model.

Resumen en español

dale-4b es un ajuste completo de Qwen3.5-4B que convierte una petición en castellano o en inglés en un único comando bash para Linux, con una explicación breve. Se entrenó con los mismos datos que dale-0.8b. En el test v2 (471 casos) acierta 410 de 471 (87,0 %), frente al 62,8 % de Qwen3.5-9B sin ajustar. Desde la versión 1.1.0, la CLI dale lo usa en la GPU (con Vulkan) cuando hay al menos 4 GiB de VRAM libre; por la ruta real de dale acierta 408 de 471 (86,6 %). También sirve para cualquier otra aplicación con llama-server, en CPU o en GPU. Usa el mismo contrato que dale-0.8b (prompt fijo, razonamiento desactivado y respuesta en JSON). Fuera de dale no hay verificador de riesgo: revisa siempre el comando antes de ejecutarlo.

The dale family

The same task, data, contract and prompt in four sizes. Scores on the public test set dale-bash-test-v2 (471 cases), all in Q4_K_M with llama-server.

Which one?

 the dale CLI, first run:
   dedicated GPU, ≥ 8 GiB free VRAM ─► you choose: 1) dale-4b (Enter, faster)  2) dale-9b (more accurate)
   dedicated GPU, ≥ 4 GiB free VRAM ─► dale-4b
   anything else (CPU, integrated GPU) ─► dale-0.8b on the CPU

 your own application (llama-server, CPU or GPU) ─► any of the four: pick by memory and accuracy

What it is and intended use

your app ─► system context ─► dale-4b ──────────► your own review ─► show / run
            (os, distro,      (llama-server,       (no risk verifier
             package manager,  CPU or GPU:          outside dale:
             installed tools)  one bash command,    you add it)
                               refuse or unsure)

• Same contract as dale-0.8b (section below): a drop-in replacement wherever you call that model through llama-server.
• Honest when unsure. The model answers refuse when a request is not a shell task and unsure when it is ambiguous, instead of inventing a command.
• The model does not rate risk. In dale, a deterministic verifier decides the risk of every command and dangerous commands need a typed confirmation. Outside dale there is none: your application must review commands before running them.

The model was trained only for this contract. It is not a general chat model.

In the dale CLI (GPU)

Since version 1.1.0, dale is one binary for CPU and GPU. On the first run it looks for a dedicated GPU through Vulkan (NVIDIA, AMD or Intel) and uses this model when a dedicated GPU has at least 4 GiB of free VRAM. With 8 GiB or more it is the default answer (Enter) of the 4B/9B question. It asks before downloading, saves the choice (dale --setup asks again) and falls back to the CPU if the GPU fails (with dale-0.8b when it is downloaded). The GPU needs the Vulkan loader (libvulkan1 / vulkan-loader / vulkan-icd-loader), which almost every desktop already has.

Safety in dale is decided by code, not by the model: caution with its reasons, danger with a confirmation word, and an honest "no" for non-commands or ambiguous requests. Real, unedited recording with dale-0.8b; the verifier is the same whatever model runs.

Times are for the whole dale process per request (start, load, prompt, generation), as the user perceives it, on one reference desktop with a 16 GB GPU: 30 requests, three warm runs each. They depend on the machine; the public figure for dale-0.8b is 1.72 s on a mid-range 2020 laptop CPU. Every process loads the weights into VRAM, which is most of the time on the GPU.

Files

sha256sum dale-4b-1.0.0-Q4_K_M.gguf

The GGUF carries general.* metadata (name, author, URL, version, license, base model). The tensors are identical to the file evaluated below; only the metadata was added.

Using it with llama-server (CPU or GPU)

Follow the contract (version 2) exactly; the model was trained on this format and nothing else. Reasoning is always disabled.

Prompt

The prompt is Qwen3.5's chat template with thinking disabled (apply_chat_template(..., add_generation_prompt=True, enable_thinking=False)), with two turns only: a fixed system prompt and the user message. No few-shot examples, no reasoning.

System prompt (exactly two lines, no trailing newline):

Turn the request into one bash command for the Linux system described in the first line of the user message.
Reply with only JSON {"type":"command|refuse|unsure","command":"...","explanation":"..."}: "command" is "" unless type is command; "explanation" is one short sentence in the language of the request.

User message: a context line, a newline, then the request (whitespace collapsed to single spaces, at most 1000 bytes):

os=linux distro=ubuntu-24.04 pm=apt tools=curl,docker,fdfind,git,jq,python3,rg,rsync,ssh,systemctl
busca en este directorio los ficheros que contienen la palabra TODO

The tools list dale checks: fd fdfind rg bat batcat eza tree ncdu fzf jq yq zip unzip 7z zstd xz unrar curl wget ip ss netstat nmcli dig nc nmap ssh rsync iptables nft ufw firewall-cmd systemctl journalctl lsof htop sudo doas hostnamectl timedatectl loginctl setfacl getfacl docker podman kubectl apt dnf yum pacman zypper apk flatpak snap git python3 make gcc ffmpeg convert magick exiftool pdftotext. Near-universal tools (coreutils, find, grep, sed, awk, tar, gzip) are not listed.

Full prompt string (line breaks shown for readability; \n are real newlines):

<|im_start|>system\n{system}<|im_end|>\n<|im_start|>user\n{context}\n{request}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n

The prompt ends with an empty <think> block, so the model starts writing the JSON straight away. Tokenize with special tokens parsed and without BOS.

Answer

Exactly one JSON object with exactly these three string keys:

{"type":"command|refuse|unsure","command":"...","explanation":"..."}

• type is command, refuse (not a shell task) or unsure (ambiguous request).
• command is one bash command (steps joined with && or |), no newlines; it is "" unless type is command.
• explanation is one short sentence in the language of the request.
• Nothing before or after the object. Treat anything else as invalid and run nothing.

Decode greedily and cap at 200 tokens.

Example

1. Install llama.cpp: a prebuilt binary from the releases, brew install llama.cpp, or the Docker image ghcr.io/ggml-org/llama.cpp:server (:server-cuda or :server-vulkan for a GPU).

2. Start the server. It downloads the model from this repository the first time:

   llama-server -hf manuelgscom/dale-4b-GGUF:Q4_K_M -ngl 99 --port 8080
   

   Or download the file yourself, check it and point to it:

   curl -LO https://huggingface.co/manuelgscom/dale-4b-GGUF/resolve/main/dale-4b-1.0.0-Q4_K_M.gguf
   sha256sum dale-4b-1.0.0-Q4_K_M.gguf     # must match the SHA-256 in Files
   llama-server -m dale-4b-1.0.0-Q4_K_M.gguf -ngl 99 --port 8080
   

   -ngl 99 puts every layer on the GPU; on CPU only, leave it out. Any llama.cpp backend works (CPU, CUDA, Vulkan, Metal…); the dale CLI uses Vulkan.

3. Send a request with one of the snippets below (Python 3, standard library only).

Raw prompt with /completion (recommended: you control the exact string):

import json, urllib.request

SYSTEM = (
    "Turn the request into one bash command for the Linux system described in the first line of the user message.\n"
    'Reply with only JSON {"type":"command|refuse|unsure","command":"...","explanation":"..."}: '
    '"command" is "" unless type is command; "explanation" is one short sentence in the language of the request.'
)
context = "os=linux distro=ubuntu-24.04 pm=apt tools=curl,docker,fdfind,git,jq,python3,rg,rsync,ssh,systemctl"
request = "busca en este directorio los ficheros que contienen la palabra TODO"

prompt = (
    f"<|im_start|>system\n{SYSTEM}<|im_end|>\n"
    f"<|im_start|>user\n{context}\n{request}<|im_end|>\n"
    "<|im_start|>assistant\n<think>\n\n</think>\n\n"
)
body = {"prompt": prompt, "n_predict": 200, "temperature": 0, "top_k": 1}
req = urllib.request.Request("http://127.0.0.1:8080/completion",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["content"])

Chat API with /v1/chat/completions: the server applies the chat template stored in the GGUF, so you must turn reasoning off with chat_template_kwargs (with an older llama.cpp build, also start the server with --jinja):

body = {
    "messages": [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"{context}\n{request}"},
    ],
    "chat_template_kwargs": {"enable_thinking": False},
    "max_tokens": 200, "temperature": 0, "top_k": 1,
}
req = urllib.request.Request("http://127.0.0.1:8080/v1/chat/completions",
                             data=json.dumps(body).encode(),
                             headers={"Content-Type": "application/json"})
print(json.load(urllib.request.urlopen(req))["choices"][0]["message"]["content"])

The chat template in this GGUF is Qwen3.5-4B's, unchanged, and it enables reasoning by default: without "enable_thinking": false the prompt does not end in the empty <think> block and the model is outside its contract.

Valid answers look like:

{"type":"command","command":"rg -l TODO","explanation":"Lista los ficheros que contienen TODO"}
{"type":"refuse","command":"","explanation":"This is a general question, not a shell task."}
{"type":"unsure","command":"","explanation":"No está claro qué hay que arreglar; indica el fichero o el problema."}

Never run the command blindly. Outside dale there is no risk verifier: add your own review step.

Evaluation

Test v2: 471 cases (339 command requests, 101 out-of-scope, 31 ambiguous; 257 in Spanish, 214 in English). Published as manuelgscom/dale-bash-test-v2.

All models in Q4_K_M, reasoning disabled, greedy decoding, each in its own llama-server. Base models use contract v1 (the system prompt plus 8 few-shot examples, which suits a model that was not fine-tuned); the dale models use contract v2 (above).

MB = 10^6 bytes.

Against dale-0.8b, case by case on the same server (McNemar): dale-4b gets 55 cases right that dale-0.8b misses, and misses 16 that dale-0.8b gets right (p < 0.0001).

Where the errors are

Fine-tuning removes format errors and the "when not to answer" errors (refusals, ambiguity). Of the 61 remaining errors, 60 are wrong commands.

By language: 87.2 % in Spanish (224/257) and 86.9 % in English (186/214).

Per-category table (correct / cases)

Methodology

• Independent test. Test v2 was written by a different, independent LLM from a written specification, validated automatically and frozen (SHA-256 fdb5564af3c153804b8780b36f9b22326340e1a57a45907765a45ab4f39991f3). No data-generation step ever read it, and the training data was decontaminated against it.
• Double judge with debate. Format and type are checked by rules. Each proposed command is judged by two independent LLM judges; if they disagree, they debate one round, and if they still disagree, the answer counts as wrong.
• Same conditions for everyone. Each model ran in its own llama-server, with greedy decoding, seed 0 and a 200-token cap. The larger dale models were measured on a later run of the same server, with dale-0.8b as a control (371/471 there, 374 in the main run).
• Verdicts have no manual review. With 471 cases the 95 % interval is about ±3 points for this model, and much wider per category.

Training

Full fine-tune of all weights of Qwen/Qwen3.5-4B, with the same recipe as dale-0.8b. Same dataset as dale-0.8b (48,841 training examples). The training data and the training recipe are not published; see the dale-0.8b card for how the data was built and the tldr-pages attribution. Quantized to Q4_K_M with an importance matrix computed on a sample of the training data.

Limitations

• bash on Linux only. No zsh, fish, PowerShell or cmd; no macOS. On Windows, use it through WSL 2.
• It can be wrong: it gets 410 of the 471 test cases right. A wrong command may look plausible. Always review a command before running it.
• It is not a security tool. The model can propose a destructive command and does not classify risk. In dale, the verifier and the confirmations do that; outside dale, your application must.
• Trained for one task with one fixed prompt. Other prompts, chat use or reasoning mode are unsupported.
• Tool choice depends on the context line: with no or wrong context it may suggest tools that are not installed.

Changes from the base model

• All weights fine-tuned for the contract above (natural language → one bash command as JSON, with refuse and unsure).
• Tokenizer and chat template unchanged from Qwen3.5-4B.
• Converted to GGUF and quantized to Q4_K_M with imatrix; general.* metadata added.

License

Apache-2.0, the same license as the base model Qwen3.5-4B. See LICENSE. The training seeds from tldr-pages are CC BY 4.0, attributed in the dale-0.8b card.