Holo4-35B-A3B-MXFP4-MoE-GGUF

Holo4-35B-A3B (Hcompany/Holo4-35B-A3B) is an Apache-2.0 open-weight 35B MoE vision-language model with 3B active parameters and a 256K context window, developed by H Company on Qwen/Qwen3.6-35B-A3B for general-purpose computer-use agents. Its training combines supervised fine-tuning with two reinforcement-learning experts merged into a single checkpoint. Paired with the hai-agents harness, it supports screenshot-based UI interaction, element localization, clicking, typing, OCR, code execution, and MCP/API tool calling. It achieves 80.8% on OSWorld ($0.05/task), 30.9% on OSWorld 2.0 ($0.61/task), 34.5% on AutomationBench ($0.02/task), and 77.6% on AndroidWorld. It also scores 30.9% on ALE-CLI, with 85.4% on MCP tool servers, 70.8% on web apps, and 64.8% on desktop apps in H Company's Agentic Task Factory suite. Available in BF16, FP8, NVFP4, and Q4 GGUF formats, it supports vLLM, SGLang, Transformers, and llama.cpp.

[!NOTE] MTP — Speculative Decoding: No

Model Files

File Name	Quant Type	File Size	File Link	Description
Holo4-35B-A3B.BF16.gguf	BF16	69.4 GB	Link	Full BF16 weights. Highest quality, largest file size.
Holo4-35B-A3B.MXFP4_MOE.gguf	MXFP4_MOE	19.8 GB	Link	Mixed precision / MoE optimized quantization. Highly efficient size for massive models, recommended if your inference engine supports it.
Holo4-35B-A3B.mmproj-bf16.gguf	mmproj-bf16	903 MB	Link	Multimodal projection file in BF16 format. Used for vision/language models.

MXFP4_MOE Test Run

Tested Holo4-35B-A3B.MXFP4_MOE.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading.

Performance Metrics

Metric	Value
Average Prompt Processing Speed	260.44 tokens/s
Average Generation Speed	215.28 tokens/s
Total Wall Time	57.2 seconds
Context Length	16,384 tokens
KV Cache (K)	`q8_0`
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition
GPU VRAM Capacity	96 GB
Peak GPU VRAM Usage	19.07 GiB
Peak GPU Utilization	88%
GPU Layers Offloaded	99 (`-ngl 99`)
Random Seed	42
Speculative Decoding	No
Test Runs	5

Test Shots

Shot	Test
Shot 1	C: Insert Node at End of Singly Linked List
Shot 2	Jogger and Train Passing Time
Shot 3	Two Pipes Filling a Tank
Shot 4	Students in Examination Rooms
Shot 5	Father and Son Ages

Run Inference

./build/bin/llama-cli \
  -m /path/to/models/Holo4-35B-A3B.MXFP4_MOE.gguf \
  -ngl 99 \
  -c 16384 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 8192 \
  -st \
  -p "Your prompt here"

Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp

Snapshot

Quantized using llama.cpp by prithivMLmods

prithivMLmods/Holo4-35B-A3B-MXFP4-MoE-GGUF