DiffusionGemma 26B A4B Instruct — MLX 4-bit

An independent MLX conversion of Google's original DiffusionGemma 26B A4B instruction-tuned checkpoint, prepared for native DarkBloom/d-inference support on Apple Silicon.

DiffusionGemma generates text through iterative block diffusion and accepts text, image and video inputs. This package retains the complete trained encoder/decoder, self-conditioning, tied embedding/output projection and vision components. It does not contain an autoregressive MTP assistant: native diffusion is the model's generation mechanism.

Artifact and conversion

• Original source: google/diffusiongemma-26B-A4B-it
• Source revision: f7f5b7f5fa82ffc52addd066915886d497f5517b
• Native architecture: diffusion_gemma / diffusion_gemma_text
• Configured context: 262,144 tokens. Hardware-tested contexts are separate below.
• Four safetensors shards; 1,647 indexed tensors derived from all 1,047 original tensor entries, with explicit affine scales/biases for 300 quantized modules.
• Affine quantization, group size 64: base 4-bit, with 236 sensitive modules using 8-bit according to the model-native predicate. Nonquantized tensors retain their original BF16 bytes. The full precision map is in config.json.
• Weight-shard size: 16,543,055,405 bytes (15.407 GiB). Runtime memory additionally includes caches, activations, diffusion workspace and allocator residency.
• Converter: MLX-VLM revision e79b0e041677ec4ca5333ba750376bb4e8c434cb, native DiffusionGemma quantization predicate, CPU conversion with MLX 0.32.2, MLX-LM 0.31.3 and Transformers 5.14.0. No fine-tuning or external draft model.

Weights were converted directly from the original Google BF16 payload, not requantized from another quantized checkpoint. Quantization is lossy; it is separate from any lossless runtime optimizations. A freshly produced conversion is not, by itself, evidence of better quality than another conversion.

Tokenizer, canonical chat template, generation configuration and multimodal processor metadata accompany the weights. SHA256SUMS.json covers the complete distribution. conversion.json records the recipe and immutable source binding; the complete per-module precision map is retained alongside the source identity.

Runtime

Use a DarkBloom build with native DiffusionGemma support. The implementation uses the encoder cache and bidirectional diffusion canvas, and exposes only committed output. Do not load this checkpoint through an ordinary Gemma autoregressive decoder or enable an unrelated speculative/MTP head.

The native generation configuration retains the original 256-token canvas, 48-step ceiling and entropy/stability policy. Benchmark results count committed non-EOS output, including first-block generation work; they do not count intermediate denoising tokens as generated output.

The benchmark executable is bound to d-inference 1e679214e5041fba4711bbdc28fb2e276f7e8025 and MLX-Swift-LM 566df4277bf1e4c8815e7b41e3e1a93cfb79cdc1. Native support is tracked in d-inference and MLX-Swift-LM. Measurements explicitly enable soft-embedding projection, the compiled diffusion sampler and the existing Gemma prefill/expert profile; throughput depends on the hardware, workload and selected runtime profile.

Validation and performance

Complete tensor/index/byte-span validation passed. All 747 nonquantized tensor entries are byte-identical to the original BF16 source. Every source file was hash-verified against its pinned manifest before and after conversion.

DarkBloom local authenticated API checks: 24/26 passed, 1 failed, 1 dependent case skipped. Coverage includes Chat/Responses, stream/nonstream, reasoning ON/OFF, weather tools and actual tool-result history, authentication and invalid-budget rejection. The non-reasoning streamed Responses required-tool request failed generation; its dependent history case was not executed.

Additional validation cases: media 21/21, expanded 36/44, whitespace 8/8. Failures are retained in validation.json; this is not a full production sign-off.

Apple M3 Ultra, 256 GiB; paged backend, prefix reuse OFF; unchanged native 256-token canvas/48-step ceiling, temperature 1, seed 341, 2,048-token output budget. One warmup plus two measured requests per row. Synthetic numbered text plus an article task, not representative agent traffic.

These are scoped local results, not hosted OpenRouter certification or a claim that every release gate is complete. Paged storage, prefix reuse, concurrency, trusted persistent restart and maximum-context behavior require their own artifact-specific validation beyond the measurements reported here.

Exact opaque-string tool-argument copying remains a known limitation in these tests. Related failures were also observed in an earlier original-BF16/reference evaluation. Ordinary weather-tool success does not establish exact argument fidelity for arbitrary strings, and this quantization is not a general fix for that model behavior. The failed streamed Responses case remains separately open.

Attribution and license

Original model: Google DeepMind. See the upstream model card and Google's license page. The Apache License 2.0 is included in LICENSE. ORIGINAL_MODEL_CARD.md retains the original source card. This derivative changes weight storage through quantization and serializes compatible MLX configuration/processor metadata; it does not claim Google endorsement. MLX-VLM and MLX-LM provide the conversion implementation; DarkBloom/d-inference provides the native serving integration.