Sagui-4B: tiny 2.4 GB local AI that thinks, sees and codes (GGUF)

Sagui-4B is a tiny open-weights model for Ollama and llama.cpp. It reasons step by step, reads images and screenshots, writes TypeScript, and calls tools, all offline. It runs on a 4 GB GPU from 2014.

Size: 2.4 GB in total: a 1.71 GB model plus a 0.67 GB vision projector. Without images, you only need the 1.71 GB file.

ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS

If Sagui saves you a subscription or a GPU upgrade, you can support the project: ☕ PayPal

Why Sagui-4B

40% smaller than the Q4_K_M it comes from, for 5 to 9 points less on TypeScript benchmarks. Thinking, vision and tool calling all stay.

A sagui is a marmoset, one of the smallest monkeys in the world, native to Brazil. Tiny, sharp-eyed and very quick.

See it working

These are unedited outputs from this exact GGUF. Prompts and raw responses are in examples/transcripts.json, and scripts/run_demos.py reproduces them. Sampling isn't seeded, so your wording will vary.

Screenshot → bug fix

Prompt: This is a screenshot from my editor. Explain the error in one sentence, then give the corrected line 4.

The error occurs because the reduce method's initial value of "0" is a string, which cannot be added to a number; it should be replaced with the numeric literal 0. return items.reduce((sum, i) => sum + i.price * i.qty, 0);

63 tokens, 26.8 s on a GTX 970.

UI screenshot → working React component

Thinking on, 2332 tokens, 174.2 s on a GTX 970. The code is unedited and the page has no CSS reset. It missed the field placeholders. Without thinking, the same prompt produced correct text and fields but ignored the inline-style request.

Thinking mode

Prompt: A train leaves at 14:35 and arrives at 17:10. How long is the trip in minutes?

To find the duration of the trip in minutes, we can calculate the difference between the arrival time and the departure time.

**Step 1: Calculate the time difference**
*   **Departure:** 14:35 (2:35 PM)
*   **Arrival:** 17:10 (5:10 PM)

First, look at the hours: $17 - 14 = 3$ hours. However, since we are subtracting minutes ($10 - 35$), we cannot simply subtract directly because 10 is less than 35. We need to borrow an hour from the arrival time.

*   Borrow 1 hour (60 minutes) from the hours:
    $17 \text{ hours } 10 \text{ minutes} = 16 \text{ hours } 70 \text{ minutes}$

Now, subtract the departure time ($14 \text{ hours } 35 \text{ minutes}$) from this adjusted time:
*   **Hours:** $16 - 14 = 2$ hours
*   **Minutes:** $70 - 35 = 35$ minutes

So, the trip duration is **2 hours and 35 minutes**.

**Step 2: Convert to minutes**
*   2 hours = $2 \times 60 = 120$ minutes.
*   Add the remaining minutes: $120 + 35 = 155$.

The trip is **155** minutes long.

3542 tokens, 254.5 s on a GTX 970, most of them reasoning. Thinking checks its work at length, so turn it off when you want speed.

Run it

Ollama, one command

ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS

Ollama, from the files

hf download sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF --local-dir sagui
cd sagui
ollama create sagui-4b -f Modelfile
ollama run sagui-4b "What is in this image? ./photo.png"

Control thinking per chat with --think or --think=false, or in the API:

curl http://localhost:11434/api/chat -d '{
  "model": "sagui-4b",
  "think": false,
  "messages": [{"role": "user", "content": "Write a debounce function in TypeScript."}]
}'

llama.cpp

llama-server -m Sagui-4B-IQ3_XXS.gguf --mmproj mmproj-Sagui-4B-F16.gguf -ngl 99 -c 16384

Tip for 4 GB GPUs

Ollama can split the model between CPU and GPU to be safe. On our GTX 970 that gave 15.0 tok/s. Forcing every layer onto the GPU gave 22.7 tok/s:

PARAMETER num_gpu 99

Add that line to the Modelfile and run ollama create again. If you run out of memory, remove it or lower num_ctx.

Files

Benchmarks

TypeScript code generation on MultiPL-E with greedy decoding. A solution passes only if it type-checks with tsc and passes its tests in Node.

These measure raw completion without thinking mode. Completions, failures and the script are in benchmarks/.

Measured on an Intel i5-10400, 16 GB RAM and an NVIDIA GTX 970 (4 GB). With Ollama's default split, about 3.4 GB of VRAM was in use with the desktop running. CPU only: 8.1 tok/s.

Limitations

• Latin scripts only. Removing other alphabets is part of how the model got this small. Asked to translate into Japanese and Russian, it answered in romanized text and got the Japanese wrong. Use the original Qwen3.5-4B for Chinese, Japanese, Korean, Russian, Arabic and similar languages.
• Heavy quantization. IQ3_XXS trades accuracy for size, and it was calibrated on TypeScript. Expect weaker results outside code than from larger builds.
• Thinking is slow on small hardware. A simple question can take thousands of tokens and several minutes.
• It makes mistakes. About half of the benchmark solutions fail. In our groupBy demo, the code ran correctly but failed tsc --strict with TS2322. Review generated code before running it, and don't use the model for medical, legal or safety-critical decisions. Vision was tested on screenshots and simple images, not on photos at scale.

FAQ

How small is a local model with vision and thinking? Sagui-4B does vision, reasoning and tool calling in 2.4 GB. Without images, it needs 1.71 GB.

Can it run on 4 GB of VRAM? Yes. Every result here comes from a 4 GB GTX 970.

Does it run without a GPU? Yes, on CPU, at 8.1 tok/s on an i5-10400.

Is it good for coding offline? It's useful for small TypeScript tasks and for explaining errors from screenshots. Larger models are more accurate, as the benchmarks above show.

Was it retrained? No. It has the same weights as Qwen3.5-4B, with a trimmed vocabulary and heavier quantization. See the recipe.

Can I use it commercially? Yes, under Apache 2.0. Keep the LICENSE and NOTICE files.

Em português

Sagui-4B é uma IA local de 2,4 GB que pensa, enxerga e programa. São 1,71 GB de modelo mais 0,67 GB do projetor de visão. Sem imagens, basta o arquivo de 1,71 GB. Roda offline com Ollama ou llama.cpp, inclusive em GPU de 4 GB, e responde em português e inglês. Foi calibrado para TypeScript.

ollama run hf.co/sysrobson/Sagui-4B-Vision-Thinking-Code-GGUF:IQ3_XXS

Gostou? Apoie o projeto e compartilhe.

Support the project

Sagui-4B is free and stays free. Donations pay for the compute and the time behind the next tiny models.

• PayPal

No money to spare? A ❤️ on this page and a post about what you built help just as much.

Credits and license

Sagui-4B is a modified version of Qwen3.5-4B by the Qwen team, converted from the GGUF by Unsloth and quantized with llama.cpp. It is licensed under Apache 2.0, and the changes are listed in NOTICE. This is an independent community project, not affiliated with or endorsed by the Qwen team or Alibaba Cloud.