Mistral Small 4 119B-A6B GGUF — Quantized by BatiAI

Mistral's unified open-weight model — reasoning + multimodal + agentic coding in one. 119B Mixture-of-Experts with only 6B active per token, so it runs at small-model speed while keeping frontier quality. Apache 2.0 (fully commercial-friendly, no gating).

Quantized directly from official Mistral weights — not a re-quant of someone else's GGUF. Signed with BatiAI metadata for BatiFlow.

Quick Start

ollama run batiai/mistral-small-4:q4

Available Quantizations

Quant	Size	RAM target	Recommended For
Q3_K_M	54GB	64GB Mac	Compact
Q4_K_M	68GB	96GB Mac	Recommended (balance)
Q5_K_M	79GB	128GB Mac	Max quality

119B total params → these are for 64GB+ Macs (M-series Max/Ultra). The 6B active means inference is fast despite the size. IQ3/IQ4 (smaller, imatrix) can be added on request.

RAM Requirements

Your Mac RAM	Q3 (54GB)	Q4 (68GB)	Q5 (79GB)
64GB	✅ tight	❌	❌
96GB	✅	✅	❌ tight
128GB	✅	✅	✅
192GB+	✅	✅	✅ comfortable

Why Mistral Small 4?

One model, three jobs — reasoning, multimodal understanding, and agentic coding unified (no model-switching).
6B active / 119B total MoE — frontier-class capability at the inference speed/cost of a small model.
Apache 2.0 — no license friction, no gating. Build commercial products freely.
Native llama.cpp support — Mistral ships official GGUF tooling; arch (mistral3) is mainstream.

Why BatiAI Quantization?

Original-source — quantized straight from Mistral's official weights, not a copy of a third-party GGUF.
BatiAI-signed — general.author: BatiAI, general.url: https://flow.bati.ai.
Mac-tuned selection — quant sizes chosen for real Apple Silicon RAM tiers.

Technical Details

Original Model: mistralai/Mistral-Small-4-119B-2603
Architecture: mistral3 MoE, 119B total / ~6B active per token
License: Apache 2.0
Quantized with: llama.cpp (Q8_0 intermediate → K-quants, --allow-requantize)
Quantized by: BatiAI
Note: YaRN config verified clean (no yarn_log_multiplier bug, unlike the earlier Medium 3.5 release).

About BatiFlow

BatiFlow — free, on-device AI automation for Mac. 5MB app, 100% local, unlimited. 60+ tools.

License

Quantized from mistralai/Mistral-Small-4-119B-2603. License: Apache 2.0.

Benchmarks

Mac 하드웨어 실측 벤치 대기 중 (bench.sh). 측정 후 자동 갱신.

batiai/Mistral-Small-4-119B-GGUF