Holo-3.1-4B-GGUF
Holo3.1: Fast & Local Computer Use Agents
Model Description
Holo3.1 is our latest family of Vision-Language Models (VLMs) for computer use agents. Building on Holo3, it expands support beyond browser and desktop automation to mobile environments, introduces native function-calling support for seamless integration with agent frameworks, and enables local deployment through optimized quantized checkpoints.
The Holo3.1 family spans model sizes from 0.8B to 35B-A3B parameters. Across computer use, UI grounding, mobile automation, and business workflows, Holo3.1 delivers strong performance while improving deployment flexibility and cost efficiency.
[!NOTE] For more information, please visit the original model card: https://huggingface.co/Hcompany/Holo-3.1-4B
Model Files
| File Name | Quant Type | File Size | File Link |
|---|---|---|---|
| Holo-3.1-4B.BF16.gguf | BF16 | 9.7 GB | Download |
| Holo-3.1-4B.F16.gguf | F16 | 9.7 GB | Download |
| Holo-3.1-4B.Q2_K.gguf | Q2_K | 2.12 GB | Download |
| Holo-3.1-4B.Q3_K_L.gguf | Q3_K_L | 2.69 GB | Download |
| Holo-3.1-4B.Q3_K_M.gguf | Q3_K_M | 2.54 GB | Download |
| Holo-3.1-4B.Q3_K_S.gguf | Q3_K_S | 2.34 GB | Download |
| Holo-3.1-4B.Q4_0.gguf | Q4_0 | 2.9 GB | Download |
| Holo-3.1-4B.Q4_K_M.gguf | Q4_K_M | 3.07 GB | Download |
| Holo-3.1-4B.Q4_K_S.gguf | Q4_K_S | 2.92 GB | Download |
| Holo-3.1-4B.Q5_0.gguf | Q5_0 | 3.43 GB | Download |
| Holo-3.1-4B.Q5_K_M.gguf | Q5_K_M | 3.51 GB | Download |
| Holo-3.1-4B.Q5_K_S.gguf | Q5_K_S | 3.43 GB | Download |
| Holo-3.1-4B.Q6_K.gguf | Q6_K | 3.99 GB | Download |
| Holo-3.1-4B.Q8_0.gguf | Q8_0 | 5.16 GB | Download |
| Holo-3.1-4B.mmproj-bf16.gguf | mmproj-bf16 | 676 MB | Download |
| Holo-3.1-4B.mmproj-f16.gguf | mmproj-f16 | 676 MB | Download |
| Holo-3.1-4B.mmproj-q8_0.gguf | mmproj-q8_0 | 367 MB | Download |
llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
FROM ghcr.io/ggml-org/llama.cpp:full
WORKDIR /app
RUN apt update && apt install -y python3-pip
RUN pip install -U huggingface_hub --break-system-packages
RUN python3 -c 'from huggingface_hub import hf_hub_download; \
repo="prithivMLmods/Holo-3.1-4B-GGUF"; \
hf_hub_download(repo_id=repo, filename="Holo-3.1-4B.Q5_K_M.gguf", local_dir="/app"); \
hf_hub_download(repo_id=repo, filename="Holo-3.1-4B.mmproj-bf16.gguf", local_dir="/app")'
CMD ["--server", \
"-m", "/app/Holo-3.1-4B.Q5_K_M.gguf", \
"--mmproj", "/app/Holo-3.1-4B.mmproj-bf16.gguf", \
"--host", "0.0.0.0", \
"--port", "7860", \
"-t", "2", \
"--cache-type-k", "q8_0", \
"--cache-type-v", "iq4_nl", \
"-c", "128000", \
"-n", "38912"]