ToPo-ToPo/gemma-4-E2B-it-qat-mlx-4bit

MLX 4-bit conversion of google/gemma-4-E2B-it-qat-q4_0-unquantized, made with mlx-vlm 0.6.3.

Provenance (self-converted)

• Source: google/gemma-4-E2B-it-qat-q4_0-unquantized (license: gemma)
• Tool: mlx-vlm 0.6.3 mlx_vlm.convert (q_bits=4, group_size=64, affine), ~5.6 bpw
• Validated: loads and translates correctly under mlx-vlm 0.6.3.

Usage

from mlx_vlm import load
model, processor = load("ToPo-ToPo/gemma-4-E2B-it-qat-mlx-4bit")

License

Derivative of Google Gemma; governed by the Gemma Terms of Use (https://ai.google.dev/gemma/terms) and Prohibited Use Policy. Converted to MLX.

⚡ MTP drafter (speculative decoding)

Use google/gemma-4-E2B-it-assistant — Google's official MTP drafter for this model. It loads directly in mlx-vlm (>= 0.6.3), needs no conversion, and speculative decoding is lossless. Drafters are size-specific and not interchangeable across Gemma 4 variants.

🔧 Patched chat template

chat_template.jinja differs from Google's Gemma 4 Canonical Chat Template (2026-07-09) by one intentional change; everything else is untouched.

The canonical template suppresses the thinking channel at the start of a normal model turn, but emits nothing after a tool response when enable_thinking is false. The model may then open a thinking channel on its own, and a quantized model sometimes writes the literal word thought into the answer. This patch gives the tool_response branch the same suppression:

{%- elif ns.prev_message_type == 'tool_response' -%}
    {%- if enable_thinking -%}
        {{- '<|channel>thought\n' -}}
    {%- else -%}
        {{- '<|channel>thought\n<channel|>' -}}
    {%- endif -%}
{%- endif -%}

Only that case changes — the other prompt paths render byte-identical to the canonical template. To get stock behaviour, replace chat_template.jinja with the one from the base model repo; the weights are unaffected.