Quantized by Austrian_Conqueror [Austriani]

GEMMA 4 · E2B

2.3B Effective · 128K Context · ik_llama.cpp “Small footprint. Big imagination.” 2.3B EFFECTIVE128K CONTEXTROLEPLAY FOCUSEDIMATRIX OPTIMIZED ◖◗Release Status All quantization variants have been computed and uploaded. The full set is available in the files tab. These files are specifically built for ik_llama.cpp.☰Available Files Files follow the naming convention: gemma-4-E2B-it-{QuantType}.gguf File TypeDescription .imatrixThe raw importance matrix file. Provided for anyone who wishes to generate their own custom quants. IQ4_KS4-bit imatrix quant, optimized for size/quality balance. IQ4_KS_R4Alternative 4-bit imatrix variant (R4). ⚡The Process I

Imatrix Optimization

Preserving the prose

To ensure the best possible quality at 4-bit, the quants (IQ4_KS, IQ4_KS_R4) were built using a freshly computed imatrix.

This allows the quantization process to prioritize the most critical weights and activation patterns, preserving the model's native creativity and narrative flow while compressing the ones that matter less. The `.imatrix` file is included in the repo for advanced users.

II

Architecture

Hybrid Attention

Gemma-4-E2B uses a dense decoder with a hybrid attention mechanism, combining sliding-window (512 tokens) and global attention. It features 8 attention heads over 1 KV head (GQA), with 35 layers in total.

Despite having only 2.3B effective parameters (5.1B total with embeddings), this architecture allows it to punch far above its weight class, especially for long-form roleplay and storytelling where context retention is key.

III

Multimodal Heritage

Text-only GGUF

The base Gemma-4-E2B is a multimodal model capable of processing Text, Image, and Audio. However, this repository ships text-only GGUF quants and does not include a vision projector (mmproj).

For vision tasks, you will need to pair these text weights with a separate mmproj-BF16.gguf projection model (compatible with the Gemma-4 architecture).

⚡Architecture & Specs Built on Google's Gemma-4-E2B-it, optimized for edge devices and local deployment. SpecificationValue Total Parameters2.3B (Effective) / 5.1B (with embeddings) ArchitectureGemma4ForConditionalGeneration Context Length128,000 tokens Layers35 AttentionHybrid Sliding-Window (512) + Global

◆  The Real-World Takeaway

With only 2.3B active parameters, this model is designed for fast local inference on almost any hardware. The reduced memory footprint of the imatrix quants prevents severe generation bottlenecks caused by RAM swapping, making it an excellent choice for edge deployment and long RP sessions.

---

Usage Note: When running with ik_llama.cpp or llama.cpp, always pass the --jinja flag so the Gemma-4 chat template is applied correctly. Without it, the model may emit malformed turns.

◖◗Credits

◆  Google DeepMind: For the original Gemma-4-E2B-it model and the UltraData training suite.

---

◆  ik_llama.cpp: For the optimized CPU inference ecosystem.

◖ · ◗