turintech/gemma-4-E2B-it-Artemis-GGUF

TurinTech models, built with Artemis

Google's Gemma 4 E2B for chat and tool calling on a CPU, from Google's quantization-aware (QAT) checkpoint.

• 2.31 GB download for the default file, gemma-4-E2B-it-Artemis-Q3_K_L.gguf.
• 3.6 GiB of RAM on an AMD Ryzen 9 5900X CPU with --cache-ram 0, up to 11.8 GiB as the default prompt cache fills.
• 369 of 400 tool calls and 219 of 250 GSM8K problems right, against 368 and 211 for Google's own Q4_0 (3.35 GB). 37 of these items also helped choose the files.

Run it

A chat model for llama.cpp's server with the chat template stored in the file; tested with llama.cpp b11617, and with b11889 for the vision projector and the MTP drafter. -hf also downloads the projector and loads it; add --no-mmproj for text only, which skips the 987 MB download.

llama-server -hf turintech/gemma-4-E2B-it-Artemis-GGUF:Q3_K_L --port 8080 -c 8192 --jinja

Request example, images, MTP, smaller files and Ollama

llama-server -m gemma-4-E2B-it-Artemis-Q3_K_L.gguf --port 8080 -c 8192 --jinja

Tested on an Apple M3 Max (36 GB) with llama.cpp Metal (b11617): loads and answers with the command above; about 115 tokens/s generation with the Q3_K_L file.

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "In two sentences, why is the sky blue?"}],
  "max_tokens": 1500
}'

-c 8192 sets an 8,192-token context; --jinja uses the chat template stored in the file. The template turns on thinking by default, so allow enough output tokens (about 1,500) or turn thinking off in the request.

# a smaller file: download it, then the same command
hf download turintech/gemma-4-E2B-it-Artemis-GGUF gemma-4-E2B-it-Artemis-Q3_K_M.gguf --local-dir .
llama-server -m gemma-4-E2B-it-Artemis-Q3_K_M.gguf --port 8080 -c 8192 --jinja

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16. The projector also holds Gemma's audio encoder; we have not tested audio input.

hf download turintech/gemma-4-E2B-it-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m gemma-4-E2B-it-Artemis-Q3_K_L.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 7 right (7 with the F16 one), against 2 for Unsloth's UD-Q2_K_XL with Unsloth's projector (see Image questions).

Faster generation with the MTP drafter

mtp-gemma-4-E2B-it-Artemis.gguf is Google's drafter for this model (gemma-4-E2B-it-qat-q4_0-unquantized-assistant, revision ebc7e1a), converted to GGUF and quantized to Q4_0. It drafts the next tokens for the main file to check and does not run on its own. With --spec-type draft-mtp, -hf downloads it as well:

llama-server -hf turintech/gemma-4-E2B-it-Artemis-GGUF:Q3_K_L --spec-type draft-mtp --port 8080 -c 8192 --jinja

The same file is in the MTP/ folder as MTP/mtp-gemma-4-E2B-it-Artemis-Q4_0.gguf. With downloaded files:

hf download turintech/gemma-4-E2B-it-Artemis-GGUF mtp-gemma-4-E2B-it-Artemis.gguf --local-dir .
llama-server -m gemma-4-E2B-it-Artemis-Q3_K_L.gguf -md mtp-gemma-4-E2B-it-Artemis.gguf --spec-type draft-mtp --port 8080 -c 8192 --jinja

On an RTX 5090 it raised generation from 360.2 to 571.5 tokens per second on our 30 prompts (see MTP). Greedy replies are not always word-for-word the same with it on: all 10 tool-call replies matched, and 1 of the 20 chat and code replies did. --mmproj and -md can be combined.

Run it with Ollama

The same files are packaged on Ollama with the Gemma 4 renderer and parser:

ollama run turintech/gemma-4-e2b-it-artemis:2.3gb
# the smallest file
ollama run turintech/gemma-4-e2b-it-artemis:1.8gb

Ollama tags are named by size: 2.3gb (the default, also latest) is the Q3_K_L file, 2.2gb is Q3_K_M, 2.1gb is IQ3_XXS and 1.8gb is IQ2_M. Each tag holds the same file, with the same SHA-256, as this repository. To import the GGUF locally instead, use Ollama's local import with the exact default file:

hf download turintech/gemma-4-E2B-it-Artemis-GGUF gemma-4-E2B-it-Artemis-Q3_K_L.gguf --local-dir .
printf 'FROM ./gemma-4-E2B-it-Artemis-Q3_K_L.gguf\nPARAMETER num_ctx 8192\n' > Modelfile
ollama create gemma-4-e2b-it-artemis -f Modelfile
ollama run gemma-4-e2b-it-artemis

This recipe follows Ollama's GGUF import instructions. The quality results on this card were measured with llama.cpp; importing the file into Ollama does not establish equivalent results or compatibility with every Ollama version.

The packages hold the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/gemma-4-E2B-it-Artemis-GGUF:Q3_K_L

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this checkpoint (hf.co/unsloth/gemma-4-E2B-it-qat-GGUF:UD-Q4_K_XL) brought its projector along and ollama show listed vision. A local package of our Q3_K_L file and mmproj-F16.gguf also listed vision, answered 8 of the 20 chart questions and returned get_weather with Paris and celsius for a tool request. To build it, add the projector as a second FROM line:

FROM ./gemma-4-E2B-it-Artemis-Q3_K_L.gguf
FROM ./mmproj-F16.gguf
PARAMETER num_ctx 8192

Choose this model

• Best for chat and tool calling on a CPU with little memory.
• Consider another file when a 2.6 GB file fits: Unsloth's UD-Q4_K_XL (2.62 GB) is closer to the original. No paired test separated ours from Google's Q4_0, and every quantized file followed fewer IFEval prompts than the original.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/gemma-4-E2B-it-Artemis-GGUF:Q3_K_L.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/gemma-4-e2b-it-artemis:<tag>.
• LM Studio's Hub entry turintech/gemma-4-e2b-it-artemis offers every model file above, without the vision projector or the companion file.

Without a tag, -hf and hf.co fetch gemma-4-E2B-it-Artemis-IQ2_M.gguf, not the default file, so always add the tag.

The vision projector and the MTP drafter are optional. The drafter is there twice, in MTP/ and at the top level, where -hf finds it.

How the files compare

"Saved" is the file-size reduction versus the BF16 original. Variant links download the exact GGUF; full filenames are in the commands and checksums. File labels are size classes, explained under "File names, sizes and SHA-256 checksums" at the end of the card.

Q3_K_L: Default. Closest to the original among our files. Loses 3 of 16 maths questions in our sample.

"Same next token" is how often a file predicts the same next token as the original. "Matching tokens (first 32)" is the share of the first 32 generated tokens that match the original's token at the same position (greedy decoding, averaged over 50 prompts). "Task accuracy change" is the difference in accuracy against the original on 310 task items, in points. This small sample does not establish equivalence. KL (in the test table) measures how far a file's next-token probabilities are from the original's; lower is closer.

Results

Our default file against Unsloth's larger file, Google's own Q4_0 and the original, in the full task suite on a Ryzen 9 5900X CPU. Same next token is measured on OpenAssistant chat text.

Unsloth's UD-Q4_K_XL is closer to the original if a 2.6 GB file fits. Every quantized file followed fewer IFEval prompts than the original. GPU memory was not measured. How it was tested.

Next-token agreement with BF16 on 640 scored positions, measured on an AMD Ryzen 9 9950X CPU with llama.cpp 0504396. BF16 agrees with itself by definition. Token agreement is not task accuracy; smaller variants drift more in longer continuations. Google's Q4_0 and bartowski's file are in the full task suite, not in this plot. The unconfirmed-checkpoint row is omitted from the plot; all results remain in the comparison table.

Evaluation details and limitations

How it was tested

Recommendation notes

Download gemma-4-E2B-it-Artemis-Q3_K_L.gguf (2.31 GB) and run it with llama.cpp's chat server, as in Run it. It predicts the same next token as the original 95.8% of the time and loses 3 of 16 maths questions in our task sample. If a 2.6 GB file fits, Unsloth's UD-Q4_K_XL is closer to the original. On a 12-thread Ryzen 9 5900X CPU it peaked at 3.6 GiB of memory with an 8,192-token context and the prompt cache off (--cache-ram 0), and at 11.8 GiB over a long benchmark with llama-server's default prompt cache. On IFEval it followed 411 of 541 prompts against 428 for the original, a drop shared by every quantized file made from the QAT checkpoint.

These files are conversions of Google's quantization-aware Gemma checkpoint. Google also publishes its own Q4_0 GGUF of it (google/gemma-4-E2B-it-qat-q4_0-gguf), which is in the full task suite below. Compare the measured results below with the named public alternatives when choosing a file.

All four files mix Google's Q4_0 tensors with lower-precision types; their labels are size classes, explained in the file names table at the end.

Use the exact filename or the :Q3_K_L tag in the commands above; without a tag, an automatic download may select a different file.

Size and quality trade-offs

Google's Gemma 4 E2B is 9.3 GB at full precision (BF16). The files here are 2.1 to 2.3 GB for the three files tested in the Ryzen runs (75 to 78% smaller) and 1.78 GB for IQ2_M (81% smaller). The Artemis Q3_K_L file predicts the same next token as the original 95.8% of the time. On a 310-item task sample none of the three Ryzen-tested files shows a statistically detectable change (the Artemis Q3_K_L file: -1.6 points, 95% range -5.6 to +2.4; it loses 3 of 16 maths questions). Below 2.6 GB, the public files we tested that come from the same checkpoint predict a different next token 45 to 55% of the time and lose 23 to 34 points on the same tasks.

If a 2.6 GB file fits the available storage, Unsloth's UD-Q4_K_XL is closer to the original (table below). These files are for when a task needs a smaller download.

Google's own Q4_0 GGUF of the same checkpoint is 3.35 GB. In the full task suite below, the Artemis Q3_K_L file is 31% smaller, its KL from the original is about a fifth of the Google file's (.0101 against .0524 on WikiText-2), it peaked at 3.6 GiB of memory against 4.4 GiB with the prompt cache off, and no paired test separated the two on any of the four tasks.

Every file, ours and public, was compared with the BF16 conversion of the same QAT original at identical llama.cpp settings (context 8192, flash attention on, F16 KV cache, greedy decoding, thinking off), on:

• KL divergence and next-token agreement over 640 positions (20 fixed documents, 32 scored positions each),
• greedy 32-token continuations on 50 prompts (10 each of code repair, maths, chat, multilingual and report excerpts),
• a 310-item task sample from ARC-Challenge, WinoGrande, HellaSwag, MMLU-Pro, GSM8K, BFCL and IFEval (64 ARC-Challenge, 64 WinoGrande, 64 HellaSwag, 70 MMLU-Pro, 16 GSM8K, 16 BFCL, 16 IFEval). This is a sample for comparing files, not a benchmark score.

Public files: unsloth/gemma-4-E2B-it-qat-GGUF revision 66a399f, mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF revision 62734e9. Measured with llama.cpp 0504396 on an AMD Ryzen 9 9950X, CPU only (8 threads). Every row comes from the same machine. No speed or memory numbers in this run.

Full task suite, AMD Ryzen 9 5900X

A larger comparison in October 2026 added Google's own Q4_0 GGUF, bartowski's Q4_K_M, full task sets and peak memory. Every file was compared with the same BF16 reference on an AMD Ryzen 9 5900X, CPU only (12 threads), with llama.cpp b11617 (14310190c). The task runs used llama-server with context 8192, flash attention on, F16 KV cache, greedy decoding, thinking off and one slot. The KL runs used llama-perplexity on 512-token chunks with 12 threads and set neither flash attention nor the KV cache type, so llama.cpp's defaults applied. Each file was compared on:

• KL divergence and next-token agreement over 40 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations.
• Native tool calling on BFCL simple (Python), all 400 tasks, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K maths, the first 250 test problems.
• MMLU-Pro, 280 questions (20 from each category), zero-shot with a direct letter answer and replies capped at 8 tokens. This is not the leaderboard protocol. A reply without a letter counts as wrong, and even the original gave no letter on 122 of 280, so these counts mix answer format with knowledge.
• IFEval, all 541 prompts, strict prompt-level accuracy with the lm-evaluation-harness v0.4.13 checks.

37 of the 1,471 task items (16 BFCL, 3 GSM8K, 2 MMLU-Pro and 16 IFEval) were also in the 310-item sample used when the files were chosen. Peak memory comes from a separate llama-server run per file with the prompt cache off (--cache-ram 0), answering the first 25 items of each task. It is the server's peak resident memory, which counts the parts of the memory-mapped model file that were read.

bartowski's file is made from google/gemma-4-E2B-it, not the QAT checkpoint, so its distance from the QAT reference says nothing about quantization loss; its task scores are comparable.

Each cell gives the items only the first file got right, the items only the second got right, and p. The run made 76 paired tests; after a Bonferroni correction for 76, the 16 differences that remain all involve mradermacher's file. Every quantized file made from the QAT checkpoint followed fewer IFEval prompts than the original: ours 410 to 415 of 541, Google's 410 and Unsloth's 419, against 428. bartowski's file from the non-QAT model followed 427. Peak memory falls less than download size, and mradermacher's file needed the least memory despite a larger download. The memory figures need --cache-ram 0. With llama-server's default 8 GiB prompt cache, which the commands on this card keep, memory grew across the full benchmark to 10.9 to 14.1 GiB per file. For the quantized files the growth was about 8.1 GiB each, close to the cache cap. The BF16 original grew by 9.5 GiB, so more of its file was read later in the run, and its 4.7 GiB peak from 25 items per task probably understates a longer run.

Public files: google/gemma-4-E2B-it-qat-q4_0-gguf revision 675cff4 (gemma-4-E2B_q4_0-it.gguf), unsloth/gemma-4-E2B-it-qat-GGUF revision 66a399f (gemma-4-E2B-it-qat-UD-Q4_K_XL.gguf), mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF revision 62734e9 (gemma-4-E2B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf) and bartowski/google_gemma-4-E2B-it-GGUF revision 81012ba (google_gemma-4-E2B-it-Q4_K_M.gguf), each checked by SHA-256. No speed numbers yet.

Smallest file (IQ2_M)

Added on 25 October 2026. gemma-4-E2B-it-Artemis-IQ2_M.gguf is 1.78 GB. It was tested on a separate confirmation panel, on an NVIDIA A100 with llama.cpp b11617 (14310190c), greedy decoding, thinking off, and the same chat template and settings for every file (context 8192 and one server slot for tool calls; context 32768 shared by four slots for GSM8K):

• Native tool calling on BFCL simple (Python) items 200 to 399, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K test items 250 to 499. A reply counts only if it finished on its own and its final answer (the last \boxed{} or the last number) is right; replies may use up to 4,096 tokens. No reply showed reasoning text, and none ran out of tokens except 18 of 250 from Unsloth's file.

These are different items, hardware and scoring from the Ryzen runs above, so compare rows within this table only. This panel includes two BF16 references: Google's non-QAT google/gemma-4-E2B-it and the QAT checkpoint these files are made from ("BF16 original (QAT)", the reference everywhere else on this card).

Each cell gives the items only the first file got right, the items only the second got right, and the two-sided p, not corrected for multiple tests. Neither paired test separated the IQ2_M file from either BF16 model (QAT or non-QAT) or from the published IQ3_XXS file on this panel; that does not show they are equivalent. Unsloth's 2.19 GB file answered 88 of the 200 tool tasks without a tool call.

On 300 Dolly prompts, the IQ2_M file's first 32 greedy tokens matched those of the non-QAT BF16 model (google/gemma-4-E2B-it) at 42.4% of positions, against 42.8% for the IQ3_XXS file, 41.8% for Google's Q4_0 and 13.1% for Unsloth's 2.19 GB file. Because the reference is the non-QAT model, these figures include the difference between the QAT checkpoint and the model it came from, not only quantization loss. Treat them as provisional: the capture tool that produced them has not completed its own qualification yet.

The panel's items had been used once before, with an older maths scorer, to compare the published E2B files; the GSM8K figures here are a corrected second scoring of the same items. The search that produced this file used separate development texts. Two of its versions were taken to this panel, which decided between them by a rule fixed before the second one was scored: release the smaller one unless it did worse than the other candidate or the published IQ3_XXS file on either task. So the panel is a confirmation of this choice, not an untouched evaluation. Its memory use and speed have not been measured.

Public files: google/gemma-4-E2B-it-qat-q4_0-gguf revision 675cff4 (gemma-4-E2B_q4_0-it.gguf) and unsloth/gemma-4-E2B-it-qat-GGUF revision 66a399f (gemma-4-E2B-it-qat-UD-Q2_K_XL.gguf), each checked by SHA-256. References: BF16 conversions of google/gemma-4-E2B-it and of google/gemma-4-E2B-it-qat-q4_0-unquantized.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

Our file got 5 questions right that Unsloth's missed and Unsloth's none that ours missed (exact McNemar p = 0.06); 20 questions are too few to separate the files. Unsloth's file stopped at the 32-token limit on 6 of the 20 questions.

Our projectors were converted from google/gemma-4-E2B-it-qat-q4_0-unquantized revision 6befbac with convert_hf_to_gguf.py --mmproj from llama.cpp b11889. Their tensors are the same, byte for byte, as those of Unsloth's mmproj-BF16.gguf and mmproj-F16.gguf (unsloth/gemma-4-E2B-it-qat-GGUF revision 66a399f); the header differs in general.* fields (names, licence, tags), in clip.use_gelu (ours only) and in one audio setting (layer-norm epsilon 1e-6, as in Google's config, against Unsloth's 1e-5).

MTP

Speed was measured with the 30 fixed prompts of our Qwen3.8 27B release (10 chat, 10 code, 10 tool calls with four tools), greedy, thinking off, 256 output tokens, two warm-ups and two passes, on one RTX 5090 with llama.cpp b11889: -ngl 99 -ngld 99 -c 8192 -np 1 --cache-ram 0 --spec-type draft-mtp and llama.cpp's default draft length. Speed is generation tokens per second from llama-server's own timings (prompt excluded), over both passes; acceptance is the share of drafted tokens the main file kept.

The drafter is converted from google/gemma-4-E2B-it-qat-q4_0-unquantized-assistant revision ebc7e1a. Our BF16 and Q8_0 conversions have the same tensors, byte for byte, as Unsloth's MTP/ files of the same precision; the headers differ in the name fields and the chat template (ours has the one in Google's current revision). Our Q4_0 is stock llama-quantize Q4_0, which keeps the token embedding in Q6_K; Unsloth's Q4_0 is smaller and has it in Q4_0. We ship our Q4_0: it was within 1.6% of the fastest of the three here.

With the drafter, greedy replies matched the replies without it on 11 of 30 prompts: all 10 tool calls and 1 of the 20 chat and code prompts. The others change wording partway through, on the same prompts whichever drafter is used, while two passes of each setting gave identical replies. That points to the batched verification rather than the drafter, but the records hold text and hashes, not logits, so the cause is untested.

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol ryzen9950x-2026-10-12

AMD Ryzen 9 9950X, 12 October 2026.

• Reference: BF16 GGUF conversion of google/gemma-4-E2B-it-qat-q4_0-unquantized revision 6befbaca7398925921802abd1f277b495b78b738 (540 text tensors checked byte for byte against the original), converted with llama.cpp commit 0504396, SHA-256 29495ea57fa7db28292024529ca5d59b24f0d8fd1f610735173b25004fa27ef9
• Text fixed: 20 fixed documents, 10 chat and 10 multilingual (quality pack v1, fuller tier); dataset: HuggingFaceH4/ultrachat_200k (test_sft) and CohereForAI/aya_dataset (test); split: test_sft (chat), test (multilingual); chunks: 20 documents, last 32 positions of each; context (tokens): up to 8192 (each document after its own preceding text); scored tokens: 640
• Tool: our quality-pack script, which reads llama.cpp logits through its C API: for each of 20 fixed documents it scores the last 32 next-token distributions after the document's own preceding text. Mean KLD is the mean over these 640 positions of KL(reference || file) over the full vocabulary, natural log, with the reference log-probabilities stored as 32-bit floats; Same top p is the share of positions where the file and the reference pick the same most likely token
• llama.cpp build: commit 0504396140d1c882f5f6ee34466a42db7ae90114
• Command: run.py --config <quality config> --candidate <file.gguf> --same-checkpoint --runtime-json <runtime> --tiers fuller --metrics-output <out.json> (our quality-pack script; it is not published)
• Hardware: AMD Ryzen 9 9950X, CPU only
• Settings: CPU only, 8 threads pinned to CPUs 0 to 7, batch 512, micro-batch 128 for the files and 512 for the reference, F16 K and V cache, flash attention on, thinking off; every file gets the reference's prompt token ids, chat template and end-of-generation ids
• KL unit: nats (natural log)
• Measured by: TurinTech measured every file in this table on the same machine against the same reference. Our three files were measured on builds whose header metadata was rewritten afterwards when they were named, so the published files differ from the measured ones by 32 to 288 bytes of metadata
• Selection: this text tier is the screening tier the published files were chosen on, so it favours them
• Source on this card: the table columns "KL", "Same next token"
• Notes: Unsloth's UD-Q2_K_XL may come from a different source checkpoint, as the card says; the IQ2_M file has no closeness measurement
• Comparison files: mradermacher: mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF at 62734e900a7a, gemma-4-E2B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf, SHA-256 7ad5b1248d53f7d1d6aac2aee31bea5c02b71dc50c484418c55e76fa31cd5d72; Unsloth: unsloth/gemma-4-E2B-it-qat-GGUF at 66a399f68ddd, gemma-4-E2B-it-qat-UD-Q4_K_XL.gguf, SHA-256 e531007218dfab990486a5de7676a6932d6ea8dea233d1f698d7c21cf8a16889; mradermacher: mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF at 62734e900a7a, gemma-4-E2B-it-qat-q4_0-unquantized.i1-IQ2_M.gguf, SHA-256 8d8c663cd6b102c8afe075f1772ea076513a7ec5b4b27272249083dc99b53c5f; mradermacher: mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF at 62734e900a7a, gemma-4-E2B-it-qat-q4_0-unquantized.i1-IQ2_S.gguf, SHA-256 82307d29c1dd90ba5f27acd7d12395edd8bc1b7658b338f7ed7361503c206e13; mradermacher: mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF at 62734e900a7a, gemma-4-E2B-it-qat-q4_0-unquantized.i1-IQ2_XXS.gguf, SHA-256 6def39965ed532e529a1f572bd01bdbb7a63f5eef04b24be66b6c035fb00afae; Unsloth: unsloth/gemma-4-E2B-it-qat-GGUF at 66a399f68ddd, gemma-4-E2B-it-qat-UD-Q2_K_XL.gguf, SHA-256 0a5bbc20f91f92da96ab4870fa71b356c45b8500a7b8b9c3e0eb48359b72da28

Protocol ryzen9950x-2026-10-12, text fixed (20 fixed documents, 10 chat and 10 multilingual (quality pack v1, fuller tier)):

Protocol ryzen5900x-2026-10-20

AMD Ryzen 9 5900X, 20 and 21 October 2026.

• Reference: BF16 GGUF conversion of google/gemma-4-E2B-it-qat-q4_0-unquantized revision 6befbaca7398925921802abd1f277b495b78b738 (text only, 9,273,528,288 bytes), converted with llama.cpp b11617, SHA-256 34b7d7e9febe8b5ce6d632901f0fc2fc4113699123ac3bbdbf1fefa54c45d59a
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 40; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 40; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD and Same top p are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 40 -t 12 -ngl 0; the reference logs were written with the same command without --kl-divergence. Reconstructed from the committed copy of our scoring script with the recorded 40 chunks, not from a record of the exact run
• Hardware: AMD Ryzen 9 5900X, CPU only, 12 threads
• Settings: CPU only; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings on the card are for the task runs)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Google, Unsloth and mradermacher files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Source on this card: the table columns "KL wiki / chat", "Same next token, wiki / chat"
• Notes: bartowski's Q4_K_M is made from the non-QAT model, so the card prints "not comparable" for it and it is not repeated here
• Comparison files: Google: google/gemma-4-E2B-it-qat-q4_0-gguf at 675cff42a74c, gemma-4-E2B_q4_0-it.gguf, SHA-256 fa401b55b07ee70a54c6dae3903c783a6e65064312529ea57175cb5f8dec6634; mradermacher: mradermacher/gemma-4-E2B-it-qat-q4_0-unquantized-i1-GGUF at 62734e900a7a, gemma-4-E2B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf, SHA-256 7ad5b1248d53f7d1d6aac2aee31bea5c02b71dc50c484418c55e76fa31cd5d72; Unsloth: unsloth/gemma-4-E2B-it-qat-GGUF at 66a399f68ddd, gemma-4-E2B-it-qat-UD-Q4_K_XL.gguf, SHA-256 e531007218dfab990486a5de7676a6932d6ea8dea233d1f698d7c21cf8a16889

Protocol ryzen5900x-2026-10-20, text wiki (WikiText-2 test):

Protocol ryzen5900x-2026-10-20, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Memory

On an AMD Ryzen 9 5900X, CPU only, with llama.cpp b11617, an 8,192-token context and the prompt cache off (--cache-ram 0), llama-server peaked at 3.6 GiB with the Q3_K_L file, 3.5 GiB with Q3_K_M and 3.4 GiB with IQ3_XXS, against 4.7 GiB for the BF16 original and 4.4 GiB for Google's Q4_0. The peak counts the parts of the memory-mapped model file that were read. The commands on this card keep llama-server's default 8 GiB prompt cache; over the full benchmark, memory then grew to 11.8 GiB with Q3_K_L, 11.7 GiB with Q3_K_M and 11.5 GiB with IQ3_XXS as the cache filled. Add --cache-ram 0 to the command to stay near the lower figures. GPU memory was measured only for the Q3_K_L file (below), and the IQ2_M file's memory has not been measured at all. The full table is in How it was tested; context length and other runtime settings also affect memory use.

On an NVIDIA RTX 5090 with llama.cpp b11889 (-ngl 99 -c 8192 -np 1 --cache-ram 0), GPU memory in use was 1.96 GiB for the Q3_K_L file alone and 2.05 GiB with the MTP drafter after the 30 speed prompts, and 3.19 GiB with mmproj-F16.gguf after the 20 chart questions.

File names, sizes and SHA-256 checksums

File names

File names end in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick a file by tag. All four files mix Q4_0 with lower-precision types, so their labels are size classes: the smallest standard llama.cpp quant whose typical bits per weight are at least the file's. The general.file_type field inside every file reads Q4_0.

The IQ2_M file is smaller than the lowest step of the reference scale these labels use (IQ3_XXS). Its label is extrapolated below that scale, by shifting llama-quantize's typical bits per weight by the same amount as at IQ3_XXS (IQ2_M about 3.19, IQ2_S about 2.99), so treat it as an approximate size class.

The earlier names were removed from the repository on 26 October 2026; the files under the names above hold the same bytes.

File checksums

Artemis

Teams use Artemis to test and improve code against their own objectives. See Artemis.

About and licence

Gemma 4 E2B is Google's smallest Gemma 4 instruction model for chat, reasoning and tool use, built for laptops and phones. These files are made from Google's quantisation-aware (QAT) checkpoint (google/gemma-4-E2B-it-qat-q4_0-unquantized), which Google trained to hold up well at low precision.

This is an independent TurinTech conversion, not affiliated with or endorsed by Google. The files are standard GGUF and run on stock llama.cpp. Changes from the original: the weights are converted to GGUF and quantized for llama.cpp; the vision and audio encoders are converted to a separate projector file, and Google's drafter model (gemma-4-E2B-it-qat-q4_0-unquantized-assistant) to the MTP file. See NOTICE.

Apache-2.0, the licence Google releases Gemma 4 under (Gemma 4 licence page); see LICENSE and NOTICE.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.