turintech/gemma-4-E4B-it-Artemis-GGUF

TurinTech models, built with Artemis

Google's Gemma 4 E4B for chat, reasoning and tool use on a CPU or a small GPU, from Google's quantization-aware (QAT) checkpoint.

• 4.22 GB download for the default file, gemma-4-E4B-it-Artemis-Q4_0.gguf.
• 3.3 GiB of GPU memory with all layers on an NVIDIA A100 80 GB; 6.8 GiB of RAM on an AMD Ryzen 9 5900X CPU with --cache-ram 0.
• 364 of 400 tool calls and 226 of 250 GSM8K problems right, against 360 and 231 for Google's own Q4_0 (5.15 GB). 37 of these items also helped choose the files.

Run it

A chat model for llama.cpp's server with the chat template stored in the file; tested with llama.cpp b11617, and with b11889 for the vision projector and the MTP drafter. -hf also downloads the projector and loads it; add --no-mmproj for text only, which skips the 992 MB download.

llama-server -hf turintech/gemma-4-E4B-it-Artemis-GGUF:Q4_0 --port 8080 -c 8192 --jinja

Request example, images, MTP, smaller files and Ollama

llama-server -m gemma-4-E4B-it-Artemis-Q4_0.gguf --port 8080 -c 8192 --jinja

Tested on an Apple M3 Max (36 GB) with llama.cpp Metal (b11617): loads and answers with the command above; about 45 tokens/s generation with the Q4_0 file.

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "In two sentences, why is the sky blue?"}],
  "max_tokens": 1500
}'

-c 8192 sets an 8,192-token context; --jinja uses the chat template stored in the file. The template turns on thinking by default, so allow enough output tokens (about 1,500) or turn thinking off in the request.

# a smaller file: download it, then the same command
hf download turintech/gemma-4-E4B-it-Artemis-GGUF gemma-4-E4B-it-Artemis-IQ3_M.gguf --local-dir .
llama-server -m gemma-4-E4B-it-Artemis-IQ3_M.gguf --port 8080 -c 8192 --jinja

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16. The projector also holds Gemma's audio encoder; we have not tested audio input.

hf download turintech/gemma-4-E4B-it-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m gemma-4-E4B-it-Artemis-Q4_0.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 9 right (8 with the F16 one), against 8 for Unsloth's UD-Q4_K_XL with Unsloth's projector (see Image questions).

Faster generation with the MTP drafter

mtp-gemma-4-E4B-it-Artemis.gguf is Google's drafter for this model (gemma-4-E4B-it-qat-q4_0-unquantized-assistant, revision 27f8d20), converted to GGUF and quantized to Q4_0. It drafts the next tokens for the main file to check and does not run on its own. With --spec-type draft-mtp, -hf downloads it as well:

llama-server -hf turintech/gemma-4-E4B-it-Artemis-GGUF:Q4_0 --spec-type draft-mtp --port 8080 -c 8192 --jinja

The same file is in the MTP/ folder as MTP/mtp-gemma-4-E4B-it-Artemis-Q4_0.gguf. With downloaded files:

hf download turintech/gemma-4-E4B-it-Artemis-GGUF mtp-gemma-4-E4B-it-Artemis.gguf --local-dir .
llama-server -m gemma-4-E4B-it-Artemis-Q4_0.gguf -md mtp-gemma-4-E4B-it-Artemis.gguf --spec-type draft-mtp --port 8080 -c 8192 --jinja

On an RTX 5090 it raised generation from 278.9 to 490.1 tokens per second on our 30 prompts (see MTP). Greedy replies are not always word-for-word the same with it on: all 10 tool-call replies matched, and 1 of the 20 chat and code replies did. --mmproj and -md can be combined.

Run it with Ollama

The same files are packaged on Ollama with the Gemma 4 renderer and parser:

ollama run turintech/gemma-4-e4b-it-artemis:4.2gb

Ollama tags are named by size: 4.2gb (the default, also latest) is the Q4_0 file, 3.8gb is IQ3_M and 3.6gb is Q3_K_M. Each tag holds the same file, with the same SHA-256, as this repository. To import the GGUF locally instead, use Ollama's local import with the exact default file:

hf download turintech/gemma-4-E4B-it-Artemis-GGUF gemma-4-E4B-it-Artemis-Q4_0.gguf --local-dir .
printf 'FROM ./gemma-4-E4B-it-Artemis-Q4_0.gguf\nPARAMETER num_ctx 8192\n' > Modelfile
ollama create gemma-4-e4b-it-artemis -f Modelfile
ollama run gemma-4-e4b-it-artemis

This recipe follows Ollama's GGUF import instructions. The quality results on this card were measured with llama.cpp; importing the file into Ollama does not establish equivalent results or compatibility with every Ollama version.

The packages hold the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/gemma-4-E4B-it-Artemis-GGUF:Q4_0

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this checkpoint (hf.co/unsloth/gemma-4-E4B-it-qat-GGUF:UD-Q4_K_XL) brought its projector along and ollama show listed vision. A local package of our Q4_0 file and mmproj-F16.gguf also listed vision, answered 8 of the 20 chart questions and returned get_weather with Paris and celsius for a tool request. To build it, add the projector as a second FROM line:

FROM ./gemma-4-E4B-it-Artemis-Q4_0.gguf
FROM ./mmproj-F16.gguf
PARAMETER num_ctx 8192

Choose this model

• Best for chat and tool calling on a CPU or a small GPU, at the size of Unsloth's UD-Q4_K_XL.
• Consider another file when you need a file under 4 GB: our IQ3_M (3.84 GB) predicts the same next token as the original 94.7% of the time. No paired test separated ours from Google's Q4_0.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/gemma-4-E4B-it-Artemis-GGUF:Q4_0.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/gemma-4-e4b-it-artemis:<tag>.
• LM Studio's Hub entry turintech/gemma-4-e4b-it-artemis offers every model file above, without the vision projector or the companion file.

Without a tag, -hf and hf.co fetch gemma-4-E4B-it-Artemis-IQ3_M.gguf, not the default file, so always add the tag.

The vision projector and the MTP drafter are optional. The drafter is there twice, in MTP/ and at the top level, where -hf finds it.

How the files compare

"Saved" is the file-size reduction versus the BF16 original. Variant links download the exact GGUF; full filenames are in the commands and checksums. File labels are size classes, explained under "File names, sizes and SHA-256 checksums" at the end of the card.

"Same next token" is how often a file predicts the same next token as the original. "Matching tokens (first 32)" is the share of the first 32 generated tokens that match the original's token at the same position (greedy decoding, averaged over 50 prompts). "Acc. Δ" is the difference in accuracy against the original on 310 task items, in points. This small sample does not establish equivalence. KL (in the test table) measures how far a file's next-token probabilities are from the original's; lower is closer.

Results

Our default file against Unsloth's file of the same size, Google's own Q4_0 and the original, in the full task suite on a Ryzen 9 5900X CPU. Same next token is measured on OpenAssistant chat text.

No paired test separated our file from Google's on any task, and Google's file got a few more GSM8K problems and IFEval prompts right. On a CPU, memory grows up to 15.1 GiB as llama-server's default 8 GiB prompt cache fills; --cache-ram 0 keeps it at 6.8 GiB. How it was tested.

Next-token agreement with BF16 on 640 scored positions, measured on an AMD Ryzen 9 9950X CPU with llama.cpp 0504396. BF16 agrees with itself by definition. Token agreement is not task accuracy; smaller variants drift more in longer continuations. Google's Q4_0 and bartowski's file are in the full task suite, not in this plot. The unconfirmed-checkpoint row and aggregate range of six files are omitted from the plot; all results remain in the comparison table.

Evaluation details and limitations

How it was tested

Recommendation notes

Download gemma-4-E4B-it-Artemis-Q4_0.gguf (4.22 GB) and run it with llama.cpp's chat server, as in Run it. It predicts the same next token as the original 98.4% of the time and tests about the same as Unsloth's UD-Q4_K_XL of the same size. To stay under 4 GB take IQ3_M (94.7%, no change on our 310-item task sample). Q3_K_M is the smallest, and its longer answers drift from the original sooner. On a 12-thread Ryzen 9 5900X CPU the Q4_0 file peaked at 6.8 GiB of memory with an 8,192-token context and the prompt cache off (--cache-ram 0), and at 15.1 GiB over a long benchmark with llama-server's default prompt cache. On IFEval it followed 449 of 541 prompts against 451 for the original.

These files are conversions of Google's quantization-aware Gemma checkpoint. Google also publishes its own Q4_0 GGUF of it (google/gemma-4-E4B-it-qat-q4_0-gguf), which is in the full task suite below. Compare the measured results below with the named public alternatives when choosing a file.

The recommended Q4_0 file is a plain Q4_0 conversion: every weight tensor is Q4_0, the same per-tensor layout as Unsloth's UD-Q4_K_XL of this checkpoint, and the two files differ in size by 64 bytes. The two smaller files mix Q4_0 with lower-precision types; their labels are size classes, explained in the file names table at the end.

Use the exact filename or the :Q4_0 tag in the commands above; without a tag, an automatic download may select a different file.

Size and quality trade-offs

Google's Gemma 4 E4B is 14.9 GB at full precision (BF16). The files here are 3.6 to 4.2 GB, 72 to 76% smaller. The Artemis Q4_0 file is the closest of the three: it predicts the same next token 98.4% of the time, and 93.9% of the first 32 tokens it writes match the original's. Under 4 GB, the Artemis IQ3_M file still predicts the same next token 94.7% of the time and scores the same on a 310-item task sample (0.0 points; this small sample does not establish equivalence), while the public files we tested below 4 GB predict a different next token 27 to 74% of the time and lose 7 to 40 points on the same 310-item sample.

At 4.2 GB the main public alternative is Unsloth's UD-Q4_K_XL; the two test about the same (tables below).

Google's own Q4_0 GGUF of the same checkpoint is 5.15 GB. In the full task suite below, the Artemis Q4_0 file is 18% smaller, its KL from the original is about a thirtieth of the Google file's (.0012 against .0374 on WikiText-2), it peaked at 6.8 GiB of memory against 7.3 GiB with the prompt cache off, and no paired test separated the two on any of the four tasks. Google's file got more GSM8K problems and IFEval prompts right (231 and 456 against 226 and 449), ours more tool calls (364 against 360), and both 107 MMLU-Pro questions.

Every file, ours and public, was compared with the BF16 conversion of the same QAT original at identical llama.cpp settings (context 8192, flash attention on, F16 KV cache, greedy decoding, thinking off), on:

• KL divergence and next-token agreement over 640 positions (20 fixed documents, 32 scored positions each),
• greedy 32-token continuations on 50 prompts (10 each of code repair, maths, chat, multilingual and report excerpts),
• a 310-item task sample from ARC-Challenge, WinoGrande, HellaSwag, MMLU-Pro, GSM8K, BFCL and IFEval (64 ARC-Challenge, 64 WinoGrande, 64 HellaSwag, 70 MMLU-Pro, 16 GSM8K, 16 BFCL, 16 IFEval). This is a sample for comparing files, not a benchmark score.

Public files: unsloth/gemma-4-E4B-it-qat-GGUF revision 8c5a9e4, TheStageAI/gemma-4-E4B-it-qat-GGUF revision 5639e0e, mradermacher/gemma-4-E4B-it-qat-q4_0-unquantized-i1-GGUF revision 4dd100a. Measured with llama.cpp 0504396 on an AMD Ryzen 9 9950X, CPU only (8 threads). Every row comes from the same machine. The sections below were measured later on newer llama.cpp builds, named in each.

Full task suite, AMD Ryzen 9 5900X

A larger comparison in October 2026 added Google's own Q4_0 GGUF, bartowski's Q4_K_M, full task sets and peak memory. Every file was compared with the same BF16 reference on an AMD Ryzen 9 5900X, CPU only (12 threads), with llama.cpp b11617 (14310190c). The task runs used llama-server with context 8192, flash attention on, F16 KV cache, greedy decoding, thinking off and one slot. The KL runs used llama-perplexity on 512-token chunks with 12 threads and set neither flash attention nor the KV cache type, so llama.cpp's defaults applied. Each file ran with the chat template stored in it.

• KL divergence and next-token agreement over 60 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations.
• Native tool calling on BFCL simple (Python), all 400 tasks, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• GSM8K maths, the first 250 test problems.
• MMLU-Pro, 280 questions (20 from each category), zero-shot with a direct letter answer and replies capped at 8 tokens. This is not the leaderboard protocol. A reply without a letter counts as wrong, and even the original gave no letter on 90 of 280 (89 to 107 across the files), so these counts mix answer format with knowledge.
• IFEval, all 541 prompts, strict prompt-level accuracy with the lm-evaluation-harness v0.4.13 checks.

37 of the 1,471 task items (16 BFCL, 3 GSM8K, 2 MMLU-Pro and 16 IFEval) were also in the 310-item sample used when the files were chosen. Peak memory comes from a separate llama-server run per file with the prompt cache off (--cache-ram 0), answering the first 25 items of each task. It is the server's peak resident memory, which counts the parts of the memory-mapped model file that were read.

bartowski's file is made from google/gemma-4-E4B-it, not the QAT checkpoint, so its distance from the QAT reference says nothing about quantization loss; its task scores are comparable.

Each cell gives the items only the first file got right, the items only the second got right, and p. The run made 76 paired tests; after a Bonferroni correction for 76, the four differences that remain are all on MMLU-Pro and all involve mradermacher's file, which got 85 of 280 right against 104 to 107 for every other file: our Q4_0, IQ3_M and Q3_K_M against it (29 / 7, 28 / 7 and 28 / 7) and it against the original (7 / 29). That file also got the most tool calls right, 373 of 400, while its KL from the original is about 300 times our Q4_0's. On IFEval our files followed 441 to 449 prompts, Unsloth's 443 and Google's 456, against 451 for the original; no IFEval test survives the correction, including our Q3_K_M against Google's Q4_0 (18 / 33, p=0.049 before correction). Our Q4_0 and Unsloth's UD-Q4_K_XL, 64 bytes apart in size, split on 30 of the 1,471 task items, where only one of the two was right.

Peak memory falls less than download size: our Q4_0 is 72% smaller than the BF16 original and needed 26% less memory, and mradermacher's file needed the least memory despite a larger download than our two smaller files. The memory figures need --cache-ram 0. With llama-server's default 8 GiB prompt cache, which the commands on this card keep, memory grew over the BFCL, GSM8K and MMLU-Pro runs to 12.5 to 18.1 GiB per file. For the quantized files the growth was about 8.3 GiB each, close to the cache cap. The BF16 original grew by 8.9 GiB, so more of its file was read later in the run, and its 9.2 GiB peak from 25 items per task probably understates a longer run.

Public files: google/gemma-4-E4B-it-qat-q4_0-gguf revision 4b4a2c1 (gemma-4-E4B_q4_0-it.gguf), unsloth/gemma-4-E4B-it-qat-GGUF revision 8c5a9e4 (gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf), mradermacher/gemma-4-E4B-it-qat-q4_0-unquantized-i1-GGUF revision 4dd100a (gemma-4-E4B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf) and bartowski/google_gemma-4-E4B-it-GGUF revision 029e941 (google_gemma-4-E4B-it-Q4_K_M.gguf), each checked by SHA-256. This suite measured no speeds; the MTP speeds are under MTP.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

The two files got the same questions right.

Our projectors were converted from google/gemma-4-E4B-it-qat-q4_0-unquantized revision 476025a with convert_hf_to_gguf.py --mmproj from llama.cpp b11889. Their tensors are the same, byte for byte, as those of Unsloth's mmproj-BF16.gguf and mmproj-F16.gguf (unsloth/gemma-4-E4B-it-qat-GGUF revision 8c5a9e4); the header differs in general.* fields (names, licence, tags), in clip.use_gelu (ours only) and in one audio setting (layer-norm epsilon 1e-6, as in Google's config, against Unsloth's 1e-5).

MTP

Speed was measured with the 30 fixed prompts of our Qwen3.8 27B release (10 chat, 10 code, 10 tool calls with four tools), greedy, thinking off, 256 output tokens, two warm-ups and two passes, on one RTX 5090 with llama.cpp b11889: -ngl 99 -ngld 99 -c 8192 -np 1 --cache-ram 0 --spec-type draft-mtp and llama.cpp's default draft length. Speed is generation tokens per second from llama-server's own timings (prompt excluded), over both passes; acceptance is the share of drafted tokens the main file kept.

The drafter is converted from google/gemma-4-E4B-it-qat-q4_0-unquantized-assistant revision 27f8d20. Our BF16 and Q8_0 conversions have the same tensors, byte for byte, as Unsloth's MTP/ files of the same precision; the headers differ in the name fields and the chat template (ours has the one in Google's current revision). Our Q4_0 is stock llama-quantize Q4_0, which keeps the token embedding in Q6_K; Unsloth's Q4_0 is smaller and has it in Q4_0. We ship our Q4_0: it was within 2.1% of the fastest of the three here.

With the drafter, greedy replies matched the replies without it on 11 of 30 prompts: all 10 tool calls and 1 of the 20 chat and code prompts. The others change wording partway through, on the same prompts whichever drafter is used, while two passes of each setting gave identical replies. That points to the batched verification rather than the drafter, but the records hold text and hashes, not logits, so the cause is untested.

Results on other hardware

Same files on other hardware

A second check on GPU with the latest llama.cpp release (v0.6.0, default CUDA build) on an NVIDIA RTX 5090: KL and same-next-token against the BF16 original on 51,200 positions of WikiText-2, all layers on the GPU. A different text from the tests above, so compare rows within this table only.

All files loaded and answered normally in llama-server.

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol ryzen9950x-2026-10

AMD Ryzen 9 9950X, October 2026.

• Reference: BF16 GGUF conversion of google/gemma-4-E4B-it-qat-q4_0-unquantized revision 476025a01dbf99361c062bbeca3d6a76bb4c4566 (text weights, 665 tensors), converted with llama.cpp commit 0504396, SHA-256 d50eefab3cd3c83bdb0782b7eb8c2485889b41d854830be1639deafc1c4da849
• Text fixed: 20 fixed documents, 10 chat and 10 multilingual (quality pack v1, fuller tier); dataset: HuggingFaceH4/ultrachat_200k (test_sft) and CohereForAI/aya_dataset (test); split: test_sft (chat), test (multilingual); chunks: 20 documents, last 32 positions of each; context (tokens): up to 8192 (each document after its own preceding text); scored tokens: 640
• Tool: our quality-pack script, which reads llama.cpp logits through its C API: for each of 20 fixed documents it scores the last 32 next-token distributions after the document's own preceding text. Mean KLD is the mean over these 640 positions of KL(reference || file) over the full vocabulary, natural log, with the reference log-probabilities stored as 32-bit floats; Same top p is the share of positions where the file and the reference pick the same most likely token
• llama.cpp build: commit 0504396140d1c882f5f6ee34466a42db7ae90114
• Command: run.py --config <quality config> --candidate <file.gguf> --same-checkpoint --runtime-json <runtime> --tiers fuller --metrics-output <out.json> (our quality-pack script; it is not published)
• Hardware: AMD Ryzen 9 9950X, CPU only
• Settings: CPU only, 8 threads pinned to CPUs 0 to 7, batch 512, micro-batch 128 for the files and 512 for the reference, F16 K and V cache, flash attention on, thinking off; every file gets the reference's prompt token ids, chat template and end-of-generation ids
• KL unit: nats (natural log)
• Measured by: TurinTech measured every file in this table on the same machine against the same reference. Our IQ3_M and Q3_K_M were measured on builds with the same byte count but a different SHA-256 from the published files (header metadata was rewritten when they were named); our Q4_0 was measured on the published bytes
• Selection: this text tier is the screening tier the published files were chosen on, so it favours them
• Source on this card: the table columns "KL", "Same next token"
• Notes: the card's row for six mradermacher files (i1-IQ1_S to i1-IQ2_M) gives ranges, not one value per file, so it is not repeated here; Unsloth's UD-Q2_K_XL may come from a different source checkpoint, as the card says
• Comparison files: Unsloth: unsloth/gemma-4-E4B-it-qat-GGUF at 8c5a9e4fd548, gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf, SHA-256 df0fd4ee07072c607c29a0a1cb4f98918426cca12f45a2776bdd6ee6d09a4de3; TheStageAI: TheStageAI/gemma-4-E4B-it-qat-GGUF at 5639e0ec22dc, gemma-4-E4B-it-qat-GGUF-M.gguf, SHA-256 9e43874d0898af032b7f89dd1b9510d3bbbcbd2933c1fed54ce7754b74b1070b; mradermacher: mradermacher/gemma-4-E4B-it-qat-q4_0-unquantized-i1-GGUF at 4dd100a2890e, gemma-4-E4B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf, SHA-256 b7c786331e582c99b1859dc70a656d6cbb605affb45a47c0f9e602a820c46779; Unsloth: unsloth/gemma-4-E4B-it-qat-GGUF at 8c5a9e4fd548, gemma-4-E4B-it-qat-UD-Q2_K_XL.gguf, SHA-256 79dde517866cfbb5c00230b530de17910fc7fc78f8827554d0e14281ce5faf03

Protocol ryzen9950x-2026-10, text fixed (20 fixed documents, 10 chat and 10 multilingual (quality pack v1, fuller tier)):

Protocol ryzen5900x-2026-10-20

AMD Ryzen 9 5900X, 20 and 21 October 2026.

• Reference: BF16 GGUF conversion of google/gemma-4-E4B-it-qat-q4_0-unquantized revision 476025a01dbf99361c062bbeca3d6a76bb4c4566 (text only, 666 tensors, 14,942,972,256 bytes), converted with llama.cpp b11617, SHA-256 b265591a2c35167d9d9a7e8e4480f386a67f09d18f19601473f86d9f6c6b62ce
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD and Same top p are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190cd0a0c831fe949ac8cbedd8abdc90695)
• Command: llama-perplexity -m <file.gguf> -f <text> --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -t 12 -ngl 0; the reference logs were written with the same command without --kl-divergence. Reconstructed from the committed copy of our scoring script, not from a record of the exact run
• Hardware: AMD Ryzen 9 5900X, CPU only, 12 threads
• Settings: CPU only; the KL command sets no flash attention or KV cache flags, so llama-perplexity's defaults applied (the flash attention and F16 KV settings on the card are for the task runs)
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Google, Unsloth and mradermacher files at the revisions below, checked each by SHA-256 and measured every file in this table with the same reference, binary and machine
• Source on this card: the table columns "KL wiki / chat", "Same next token, wiki / chat"
• Notes: bartowski's Q4_K_M is made from the non-QAT model, so the card prints "not comparable" for it and it is not repeated here
• Comparison files: Google: google/gemma-4-E4B-it-qat-q4_0-gguf at 4b4a2c1d584b, gemma-4-E4B_q4_0-it.gguf, SHA-256 676c35070db6dbe52f93e9c864ee0fba4eddea94b9c875d9cb10daff453fbaee; mradermacher: mradermacher/gemma-4-E4B-it-qat-q4_0-unquantized-i1-GGUF at 4dd100a2890e, gemma-4-E4B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf, SHA-256 b7c786331e582c99b1859dc70a656d6cbb605affb45a47c0f9e602a820c46779; Unsloth: unsloth/gemma-4-E4B-it-qat-GGUF at 8c5a9e4fd548, gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf, SHA-256 df0fd4ee07072c607c29a0a1cb4f98918426cca12f45a2776bdd6ee6d09a4de3

Protocol ryzen5900x-2026-10-20, text wiki (WikiText-2 test):

Protocol ryzen5900x-2026-10-20, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Protocol rtx5090-2026-10-12

One NVIDIA RTX 5090, 12 October 2026.

• Reference: BF16 conversion of google/gemma-4-E4B-it-qat-q4_0-unquantized revision 476025a0 made with llama.cpp v0.6.0
• Text wiki: WikiText-2 test, first 100 chunks; dataset: Salesforce/wikitext; config: wikitext-2-raw-v1; split: test; chunks: 100; context (tokens): 512
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD and Same top p are llama.cpp's own summary lines; the first 100 chunks of the text
• llama.cpp build: v0.6.0 (commit d812350), default CUDA build
• Hardware: one NVIDIA RTX 5090, all layers on the GPU
• Settings: all layers on the GPU
• KL unit: nats (natural log)
• Measured by: TurinTech measured every file in this table in one run against the same reference
• Selection: a different text from the other tables, so compare rows within this table only
• Source on this card: the table columns "KL to the original", "Same next token"
• Comparison files: Unsloth: unsloth/gemma-4-E4B-it-qat-GGUF at 8c5a9e4fd548, gemma-4-E4B-it-qat-UD-Q4_K_XL.gguf, SHA-256 df0fd4ee07072c607c29a0a1cb4f98918426cca12f45a2776bdd6ee6d09a4de3; mradermacher: mradermacher/gemma-4-E4B-it-qat-q4_0-unquantized-i1-GGUF at 4dd100a2890e, gemma-4-E4B-it-qat-q4_0-unquantized.i1-IQ3_XXS.gguf, SHA-256 b7c786331e582c99b1859dc70a656d6cbb605affb45a47c0f9e602a820c46779

Protocol rtx5090-2026-10-12, text wiki (WikiText-2 test, first 100 chunks):

Memory

On an AMD Ryzen 9 5900X, CPU only, with llama.cpp b11617, an 8,192-token context and the prompt cache off (--cache-ram 0), llama-server peaked at 6.8 GiB with the Q4_0 file, 6.4 GiB with IQ3_M and 6.2 GiB with Q3_K_M, against 9.2 GiB for the BF16 original and 7.3 GiB for Google's Q4_0, answering the first 25 items of each benchmark task. The peak counts the parts of the memory-mapped model file that were read. The commands on this card keep llama-server's default 8 GiB prompt cache; over the BFCL, GSM8K and MMLU-Pro runs, memory then grew to 15.1 GiB with Q4_0, 14.8 GiB with IQ3_M and 14.5 GiB with Q3_K_M as the cache filled. Add --cache-ram 0 to the command to stay near the lower figures. The full table is in How it was tested.

An earlier measurement gave the GPU figures: peak memory of llama-server with the command above (--port 8080 -c 8192 --jinja) answering 10 short chat prompts, llama.cpp b11574 on an NVIDIA A100 80 GB and its host.

llama.cpp keeps part of this model in system memory even with every layer on the GPU, so GPU memory is the same for all three files and the difference shows in system memory. Those 10 short prompts read less text than the benchmark above, and the CPU-only figures there are lower (5.1 against 6.8 GiB for Q4_0). System memory also grows as the default prompt cache fills. Context length and other runtime settings affect memory use too.

On an NVIDIA RTX 5090 with llama.cpp b11889 (-ngl 99 -c 8192 -np 1 --cache-ram 0), GPU memory in use was 3.31 GiB for the Q4_0 file alone and 3.41 GiB with the MTP drafter after the 30 speed prompts, and 4.55 GiB with mmproj-F16.gguf after the 20 chart questions.

File names, sizes and SHA-256 checksums

File names

File names end in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick a file by tag. The Q4_0 file is a plain Q4_0 conversion: every weight tensor is Q4_0. The other two files mix Q4_0 with lower-precision types, so their labels are size classes: the smallest standard llama.cpp quant whose typical bits per weight are at least the file's. The general.file_type field inside every file reads Q4_0.

The earlier names were removed from the repository on 26 October 2026; the files under the names above hold the same bytes.

File checksums

Artemis

Teams use Artemis to test and improve code against their own objectives. See Artemis.

About and licence

Gemma 4 E4B is Google's small instruction model for chat, reasoning and tool use, built for laptops and phones. These files are made from Google's quantisation-aware (QAT) checkpoint (google/gemma-4-E4B-it-qat-q4_0-unquantized), which Google trained to hold up well at low precision.

This is an independent TurinTech conversion, not affiliated with or endorsed by Google. The files are standard GGUF and run on stock llama.cpp. Changes from the original: the weights are converted to GGUF and quantized for llama.cpp; the vision and audio encoders are converted to a separate projector file, and Google's drafter model (gemma-4-E4B-it-qat-q4_0-unquantized-assistant) to the MTP file. See NOTICE.

Apache-2.0, the licence Google releases Gemma 4 under (Gemma 4 licence page); see LICENSE and NOTICE.

When serving these files to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.