Zen5
Canonical default of the Zen5 family. Multimodal sparse MoE (image + text in → text out) with 35B total / 3B active parameters per token, 256K context. The everyday Zen5 model — agentic-trained, fast at scale, frontier-quality vision-language reasoning at a 3B-active compute budget.
Repackaged from Qwen/Qwen3.6-35B-A3B (apache-2.0, Alibaba Qwen) — quantized to GGUF from the abliterated huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated variant. Not trained from scratch — a permissively-licensed redistribution for the OSS-clean Zen model line.
Part of the canonical Zen5 ladder:
| SKU | Hardware fit | This repo |
|---|---|---|
zen5-flash |
anything (4 GB VRAM) | zen-5-flash-gguf |
zen5-mini |
32 GB | zen-5-mini-gguf |
zen5 (default) |
24 GB+ VRAM (Q4_K) | ← you are here |
zen5-pro |
Mac M4 Max / DGX Spark / H100 80GB | zen-5-pro-gguf |
zen5-max |
Mac Studio M3 Ultra 512GB / 8x H100 | zen-5-max-gguf |
Files
| File | Format |
|---|---|
main GGUF (*-Q4_K.gguf) |
GGUF Q4_K (text + vision) |
mmproj-model-f16.gguf |
multimodal vision projector — load alongside the main GGUF for image input |
Run
Hosted via the Hanzo gateway (api.hanzo.ai) as zen5.
Local with llama.cpp (CLI / server) or zen5-engine:
hf download zenlm/zen-5-gguf --local-dir gguf
MAIN=$(ls gguf/*-Q4_K.gguf | head -1)
# text-only chat
llama-cli -m "$MAIN" -p "Explain MoE inference."
# vision-language (image input)
llama-cli -m "$MAIN" \
--mmproj gguf/mmproj-model-f16.gguf \
--image path/to/screenshot.png \
-p "Describe this UI and propose a fix."
License
apache-2.0. Upstream: Qwen/Qwen3.6-35B-A3B by Alibaba Qwen; abliterated variant by huihui-ai. This repository redistributes a quantized derivative under the same license.