Gemma 4 31B — Oxidize AL Quants

Oxidize AL-family GGUF quants of google/gemma-4-31B-it.

Quantized with oxidize-quantize from Unsloth BF16 source weights (unsloth/gemma-4-31B-it-GGUF) — lossless requant at the weight level (no quality loss from a prior quant).

Disclaimer

This quant type is experiemental

Files

File	Type	~Size	Bits	When to use
`gemma-4-31B-it-AL5.gguf`	AL5	17 GB	~4.5	Default — same block size as Q4_0, lower RMSE

What is AL?

Oxidize AL quants are custom ggml types (IDs 240–243) that use multi-seed MSE-optimal per-block scales at the same block layouts as Q4_0 / Q5_0 / Q8_0. AL5 decodes identically to Q4_0 bitstream layout but encodes with lower reconstruction error. Implementation: al_family.rs.

Quick start — llama.cpp

huggingface-cli download freakyskittle/gemma-4-31B-it-AL-GGUF gemma-4-31B-it-AL5.gguf --local-dir .

./llama-cli -m gemma-4-31B-it-AL5.gguf \
  -p "<|turn>user\nHello<turn|>\n<|turn>model\n<|channel>final<|message|>" \
  -n 256 --temp 0.7

Quick start — oxidize-cpp (CPU)

git clone https://github.com/Zapdev-labs/oxidize.git && cd oxidize/oxidize-cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release && cmake --build build -j

./build/oxidize-cpp --model gemma-4-31B-it-AL5.gguf \
  --prompt "Hello" --max-tokens 128 --auto

Quick start — oxidize-c (CUDA)

git clone https://github.com/Zapdev-labs/oxidize.git && cd oxidize/oxidize-c
make cuda

./oxidize-c-cuda --model gemma-4-31B-it-AL5.gguf \
  --prompt "<|turn>user\nHello<turn|>\n<|turn>model\n" --max-tokens 128

Reproduce these quants

cargo build -p oxidize-quantize --release
./target/release/oxidize-quantize \
  --input gemma-4-31B-it-BF16-00001-of-00002.gguf \
  --output gemma-4-31B-it-AL5.gguf \
  --target AL5 --threads 96

Remote pipeline: scripts/gemma4_31b_al_remote.sh

License

Apache 2.0 (per Gemma 4 license).

freakyskittle/gemma-4-31B-it-AL-GGUF