grug 35B v2 for mlx-serve (4-bit)

An Apache-2.0 MLX affine quantization of ProCreations/grug-35b-v2, matching the base model's license.

This is a text-only pack; it does not include the source model's vision assets.

Size and quantization

Safetensors payload: 19.51 GB / 18.17 GiB. Affine 4-bit quantization uses group size 64. MoE router gates and shared-expert gates are retained at 8-bit; norms, scales, and biases remain BF16.

Benchmarks

Benchmarks pending a reproducible rerun.

Usage

Requires mlx-serve v26.10.2 or later.

mlx-serve serve --model pgmharikrishnan/grug-35b-v2-MLX-4bit