DeskForge-Qwen3.5-4B GGUF
GGUF conversions of DeskForge v1.1. Choose one decoder and pair it with the shared F16 vision encoder/projector. Quantizations are independently derived from the original-weight F16 conversion.
| File | Component | Quantization | Size | Status |
|---|---|---|---|---|
| DeskForge-Qwen3.5-4B-Q4_K_M.gguf | decoder | Q4_K_M | 2,708,805,344 bytes | Available |
| DeskForge-Qwen3.5-4B-Q5_K_M.gguf | decoder | Q5_K_M | 3,074,987,744 bytes | Available |
| DeskForge-Qwen3.5-4B-Q8_0.gguf | decoder | Q8_0 | 4,482,404,064 bytes | Available |
| DeskForge-Qwen3.5-4B-F16.gguf | decoder | F16 | 8,424,394,464 bytes | Available |
| mmproj-DeskForge-Qwen3.5-4B-F16.gguf | vision | F16 | 672,424,192 bytes | Available |
Usage
Use llama.cpp at 0ec4027.
This configuration runs text inference on the GPU and vision on the CPU:
llama-server -m DeskForge-Qwen3.5-4B-Q4_K_M.gguf \
--mmproj mmproj-DeskForge-Qwen3.5-4B-F16.gguf \
-ngl 99 -c 4096 -np 1 --jinja --reasoning off \
--image-min-tokens 64 --image-max-tokens 2048 --no-mmproj-offload \
--host 127.0.0.1 --port 8080
Use system_prompt.txt as the system message. Send the screenshot first, followed by the task in the source format:
Task: Open the File menu.
Actions already taken:
(none -- this is the first step)
Next action:
Use greedy decoding with at most 128 output tokens and screenshots capped at 2,097,152 pixels. Output coordinates are fractions in [0,1]. Generated actions are text for a controller to interpret. The original chat template is embedded and provided as chat_template.jinja.
Source and license
Apache-2.0. Original licenses, attributions and conversion changes are in LICENSE, ATTRIBUTION.md and CHANGES.md. See REPRODUCE.md, runtime-info.json and model-manifest.json for commands, pinned provenance and file SHA256 hashes.