gemma-4-12B-coder-fable5-composer2.5-v1-mxfp8-mlx
MLX quantization of interpolators/gemma-4-12B-coder-fable5-composer2.5-v1-bf16 for Apple Silicon.
Note — quantized via mlx-lm with
model_typeset togemma4. The source declaresmodel_type: gemma4_unified(Gemma4UnifiedForConditionalGeneration), which current mlx-lm does not map; it was converted by settingmodel_typetogemma4(text configgemma4_text), which loads on mlx-lm's Gemma 4 path. Any vision/multimodal weights from the unified checkpoint are not included — this is a text-only MLX model. Post-quantization generation was sanity-checked (correct arithmetic + reasoning, stops on EOS).
Variant: Block float MX FP8
Disk size: 11742 MB
Quantized by: sahilchachra
Benchmark results
Evaluated on Apple M5 Pro with MLX. Model loaded once; performance and quality measured in a single pass.
Performance
| This model | FP16 baseline | |
|---|---|---|
| Peak memory (GB) | N/A | N/A |
| Disk size (MB) | 11742 | 22842 |
Quality
No quality benchmarks recorded for this variant.
Usage
pip install mlx-lm
from mlx_lm import load, generate
model, tokenizer = load("sahilchachra/gemma-4-12B-coder-fable5-composer2.5-v1-mxfp8-mlx")
response = generate(model, tokenizer, prompt="Your prompt here", max_tokens=256, verbose=True)
All variants in this collection
| Model | Variant |
|---|---|
| sahilchachra/gemma-4-12B-coder-fable5-composer2.5-v1-mxfp4-mlx | Block float MX FP4 |
| sahilchachra/gemma-4-12B-coder-fable5-composer2.5-v1-mxfp8-mlx | Block float MX FP8 ← this model |
Notes
- Requires Apple Silicon (M1 or later) with MLX
- Benchmarks run on Apple M5 Pro, 24 GB unified memory
- License: see interpolators/gemma-4-12B-coder-fable5-composer2.5-v1-bf16 for the original model's license
Original model
See interpolators/gemma-4-12B-coder-fable5-composer2.5-v1-bf16 for full model details and intended use.