turintech/gemma-4-12B-it-Artemis-GGUF

TurinTech models, built with Artemis

Google's Gemma 4 12B for chat, reasoning and tool use on one GPU, from Google's quantization-aware (QAT) checkpoint.

• 6.72 GB download for the default file, gemma-4-12B-it-Artemis-Q4_0.gguf.
• Peak GPU memory 8.4 GiB on an NVIDIA A100 80 GB with the command below, and 6.5 to 6.7 GiB of process memory.
• On chat text it picks the original's next token 90.3% of the time, against 77.8% for Google's own Q4_0 (6.98 GB).

Run it

A chat model for llama.cpp's server with the chat template stored in the file; tested with llama.cpp b11617, and with b11889 for the vision projector and the MTP drafter. -hf also downloads the projector and loads it; add --no-mmproj for text only, which skips the 175 MB download.

llama-server -hf turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0 --port 8080 -c 8192 --jinja

Request example, images, MTP, settings and download

llama-server -m gemma-4-12B-it-Artemis-Q4_0.gguf --port 8080 -c 8192 --jinja

curl http://localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "In two sentences, why is the sky blue?"}],
  "max_tokens": 1500
}'

-c 8192 sets an 8,192-token context; --jinja uses the chat template stored in the file. Leave the server running and send the request from a second terminal. The template turns on thinking by default: the server returns it in message.reasoning_content and the answer in message.content, so allow enough output tokens (about 1,500), or add "chat_template_kwargs": {"enable_thinking": false} to the request to turn it off. The hf command (pip install -U huggingface_hub) downloads the file on its own:

hf download turintech/gemma-4-12B-it-Artemis-GGUF gemma-4-12B-it-Artemis-Q4_0.gguf --local-dir .

Images

The vision projector is a separate file, converted from the original weights. mmproj-F16.gguf is the same projector in F16. The projector also holds the audio input layers; we have not tested audio input.

hf download turintech/gemma-4-12B-it-Artemis-GGUF mmproj-BF16.gguf --local-dir .
llama-server -m gemma-4-12B-it-Artemis-Q4_0.gguf --mmproj mmproj-BF16.gguf --port 8080 -c 8192 --jinja

Send the image as a data URL in an image_url part of the user message. On 20 chart questions this file with the BF16 projector got 13 right (12 with the F16 one), against 13 for Unsloth's UD-Q4_K_XL with Unsloth's projector (see Image questions).

Faster generation with the MTP drafter

mtp-gemma-4-12B-it-Artemis.gguf is Google's drafter for this model (gemma-4-12B-it-qat-q4_0-unquantized-assistant, revision 1893406), converted to GGUF and quantized to Q4_0. It drafts the next tokens for the main file to check and does not run on its own. With --spec-type draft-mtp, -hf downloads it as well:

llama-server -hf turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0 --spec-type draft-mtp --port 8080 -c 8192 --jinja

The same file is in the MTP/ folder as MTP/mtp-gemma-4-12B-it-Artemis-Q4_0.gguf. With downloaded files:

hf download turintech/gemma-4-12B-it-Artemis-GGUF mtp-gemma-4-12B-it-Artemis.gguf --local-dir .
llama-server -m gemma-4-12B-it-Artemis-Q4_0.gguf -md mtp-gemma-4-12B-it-Artemis.gguf --spec-type draft-mtp --port 8080 -c 8192 --jinja

On an RTX 5090 it raised generation from 160.1 to 334.6 tokens per second on our 30 prompts (see MTP). Greedy replies are not always word-for-word the same with it on: all 10 tool-call replies matched, and 1 of the 20 chat and code replies did. --mmproj and -md can be combined.

Ollama

ollama run turintech/gemma-4-12b-it-artemis:q4_0

The package holds the main file only. For images, use Ollama 0.40.11 or newer and pull from Hugging Face, which brings the repository's projector along:

ollama run hf.co/turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0

With Ollama 0.40.11 on 26 October, the same pull from Unsloth's repository of this checkpoint (hf.co/unsloth/gemma-4-12B-it-qat-GGUF:UD-Q4_K_XL) brought its projector along and ollama show listed vision. A local package of our Q4_0 file and mmproj-F16.gguf (two FROM lines, PARAMETER num_ctx 8192) also listed vision, answered 13 of the 20 chart questions and returned get_weather with Paris and celsius for a tool request.

Questions, problems or a model to consider for a smaller version? Reply in our feedback thread.

Choose this model

• Best for chat and tool calling on one GPU: 190 of 200 tool calls and 240 of 250 maths problems right, against 190 and 238 for Google's Q4_0.
• Consider another file when you want the closest file on Wikipedia text: Unsloth's UD-Q4_K_XL (6.72 GB) is slightly closer there (KL .147 against .156). No paired test separated these files on tool calls or maths.

Files

Every file we publish for this model. Sizes are download sizes, not memory.

• With llama.cpp or a Hugging Face pull, add the file's quant label as the tag: -hf turintech/gemma-4-12B-it-Artemis-GGUF:Q4_0.
• The Ollama tag runs our package, which holds the same file: ollama run turintech/gemma-4-12b-it-artemis:<tag>.
• LM Studio's Hub entry turintech/gemma-4-12b-it-artemis offers every model file above, without the vision projector or the companion file.

The vision projector and the MTP drafter are optional. The drafter is there twice, in MTP/ and at the top level, where -hf finds it.

How the files compare

"Saved" is the file-size reduction versus the BF16 original (23.83 GB). Measured results, and comparisons with the original and public files, are under How it was tested.

Results

Our file against Unsloth's file of the same size, Google's own Q4_0 and the original, on an NVIDIA A100. The tool-call and maths items were kept out of development and used in a final check fixed in advance.

No paired test separated these files on tool calls or maths. How it was tested.

Evaluation details and limitations

How it was tested

Every file was compared with the BF16 conversion of the same QAT original (google/gemma-4-12B-it-qat-q4_0-unquantized revision b6ed862, text weights) on an NVIDIA A100 80 GB with llama.cpp b11617 (14310190c), greedy decoding, thinking off, one shared chat template (Google's, from the original) and the original's BOS setting for every file.

• KL divergence and same next token over 60 chunks of 512 tokens from each of two human-written texts: the WikiText-2 test set (Wikipedia) and OpenAssistant OASST1 validation conversations. These texts were used while developing this file, so they are not an independent test.
• Native tool calling: BFCL v3 simple Python, tasks 200 to 399, through llama-server's OpenAI-style tools API, scored with BFCL's own checker. A reply with no tool call counts as wrong.
• Maths: GSM8K test problems 250 to 499, rerun with a corrected protocol: a 4,096-token limit, a reply counts only if it finished on its own, and the answer is the last \boxed{} (else the last number) of the final reply. An earlier 768-token run, which also scored unfinished replies, gave 240 for this file, 238 for the original, 239 for Unsloth's file and 236 for Google's.

The tool-calling and maths items were kept out of development. They were then used in a final check, fixed in advance, that compared this file with the previous candidate and could have selected that one instead; it did not. They are therefore not an independent test made after the choice.

Bold marks the best displayed value for each metric among the rows shown, ties included; it is not a significance claim. Unsloth's file has the same overall tensor-type counts as ours and is 64 bytes larger; it is closer to the original on WikiText-2 and ours is closer on the chat text.

Each cell gives the items only the first file got right, the items only the second got right, and p. No paired test established a difference. These samples are small and do not establish equivalence.

Public files: google/gemma-4-12B-it-qat-q4_0-gguf revision 29d0977 (gemma-4-12b-it-qat-q4_0.gguf), unsloth/gemma-4-12B-it-qat-GGUF revision 980b060 (gemma-4-12B-it-qat-UD-Q4_K_XL.gguf, the main file only; its optional MTP file was not used).

Limitations: images were checked on 20 chart questions only (see Image questions below), and audio not at all. Tested with thinking off; thinking mode and long contexts were not evaluated. Calibrated on OpenAssistant OASST1 training conversations. The importance matrix is not distributed.

Image questions

A quick check that the projector works with our files: the first 20 ChartQA test questions (human-written) from the frozen item list of our Qwen3.8 27B release, with lmms-eval's prompt (answer with a single word or phrase), greedy, thinking off, up to 32 output tokens, scored by relaxed accuracy (a number within 5% of the answer, otherwise an exact match ignoring case). llama.cpp b11889 (d854d98) with CUDA on one NVIDIA RTX 5090, -ngl 99 -c 8192 -np 1 --cache-ram 0; Unsloth's file ran with Unsloth's projector.

Unsloth's file got one question right that ours missed, and ours none that Unsloth's missed (exact McNemar p = 1.0); 20 questions are too few to separate the files. This model's projector has no vision transformer, only the patch embedding that feeds image patches to the main model. b11889's converter writes that tensor in F16 in an F16 projector; loaded that way, our main file returned empty replies to all 20 questions, so our mmproj-F16.gguf keeps it in F32, as Unsloth's does (our BF16 file already did).

Our projectors were converted from google/gemma-4-12B-it-qat-q4_0-unquantized revision b6ed862 with convert_hf_to_gguf.py --mmproj from llama.cpp b11889. Their tensors are the same, byte for byte, as those of Unsloth's mmproj-BF16.gguf and mmproj-F16.gguf (unsloth/gemma-4-12B-it-qat-GGUF revision 980b060), except that the F16 patch embedding, kept in F32 in both, differs by at most 3e-8 (ours is the exact BF16 value).

MTP

Speed was measured with the 30 fixed prompts of our Qwen3.8 27B release (10 chat, 10 code, 10 tool calls with four tools), greedy, thinking off, 256 output tokens, two warm-ups and two passes, on one RTX 5090 with llama.cpp b11889: -ngl 99 -ngld 99 -c 8192 -np 1 --cache-ram 0 --spec-type draft-mtp and llama.cpp's default draft length. Speed is generation tokens per second from llama-server's own timings (prompt excluded), over both passes; acceptance is the share of drafted tokens the main file kept.

The drafter is converted from google/gemma-4-12B-it-qat-q4_0-unquantized-assistant revision 1893406. Our BF16 and Q8_0 conversions have the same tensors, byte for byte, as Unsloth's MTP/ files of the same precision; the headers differ in the name fields and the chat template (ours has the one in Google's current revision). Our Q4_0 is stock llama-quantize Q4_0, which keeps the token embedding in Q6_K; Unsloth's Q4_0 is smaller and has it in Q4_0. We ship our Q4_0: it was the fastest of the three here.

With the drafter, greedy replies matched the replies without it on 11 of 30 prompts: all 10 tool calls and 1 of the 20 chat and code prompts. The others change wording partway through, on the same prompts whichever drafter is used, while two passes of each setting gave identical replies. That points to the batched verification rather than the drafter, but the records hold text and hashes, not logits, so the cause is untested.

Closeness to the original

The closeness numbers from the tables above, in one fixed layout with the protocol behind each table.

Protocols and standard tables

• Mean KLD: the mean KL divergence between the reference's and the file's next-token distributions, computed and averaged as each protocol's Tool line says. Lower is closer.
• 99th-percentile KLD: the 99th percentile of the per-token values. Lower is closer.
• Same top p: llama.cpp's name for top-1 agreement, the share of scored tokens where the file and the reference pick the same most likely token. Higher is closer.
• PPL and PPL ratio: the file's perplexity on the text, and that perplexity divided by the reference's (1 means the same). They measure prediction of the text, not closeness.

n/a means the card's tables or our run records do not have that value.

Protocol a100-2026-10-24

One NVIDIA A100 80 GB, 24 October 2026.

• Reference: BF16 GGUF conversion of google/gemma-4-12B-it-qat-q4_0-unquantized revision b6ed86275a6a5735884e208bfed95b445a684ca2 (text weights, 667 tensors, 23,832,066,176 bytes)
• Text wiki: WikiText-2 test; dataset: Salesforce/wikitext at b08601e04326c79dfdd32d625aee71d232d685c3; config: wikitext-2-raw-v1; split: test; chunks: 60; context (tokens): 512; text SHA-256: bbf94c53a05abe9ee670d3b6343608095822c85e26de37c70b24fc571964574a
• Text chat: OpenAssistant validation conversations (English threads as User/Assistant text); dataset: OpenAssistant/oasst1 at fdf72ae0827c1cda404aff25b6603abec9e3399b; split: validation; chunks: 60; context (tokens): 512; text SHA-256: 0a3499a7eaead8f4a40655a868e7faee1ee664e258d09a77cd8f3d97216fc8b7
• Tool: llama-perplexity --kl-divergence against the reference's saved log-probabilities, chunks of 512 tokens; Mean KLD and Same top p are llama.cpp's own summary lines
• llama.cpp build: b11617 (commit 14310190c)
• Command: llama-perplexity -m <file.gguf> --override-kv tokenizer.ggml.add_bos_token=bool:true --kl-divergence-base <reference-text.kld> --kl-divergence -c 512 --chunks 60 -ngl 99 -t 16; the reference logs were written with -f <text> and the same flags without --kl-divergence
• Hardware: one NVIDIA A100 80 GB, all layers on the GPU
• Settings: beginning-of-sequence token on for every file, as in the original; flash attention and KV cache type not set on the command line
• KL unit: nats (natural log)
• Measured by: TurinTech downloaded the Google and Unsloth files at the revisions below and measured every file in this table with the same frozen scoring script, reference logs, binary and GPU
• Selection: these texts were used while developing this file, so they are not an independent test
• Source on this card: the table columns "KL wiki / chat", "Same next token, wiki / chat"
• Comparison files: Unsloth: unsloth/gemma-4-12B-it-qat-GGUF at 980b060c40a8, gemma-4-12B-it-qat-UD-Q4_K_XL.gguf, SHA-256 90fd44e29e0d7cffeb0fd00dc73cfdab9ed0b0e95306ecf7821ea634c940c370; Google: google/gemma-4-12B-it-qat-q4_0-gguf at 29d097773436, gemma-4-12b-it-qat-q4_0.gguf, SHA-256 93567e57a8fe10b23569b9d9ec38cd005deedf71e29477c421a4b83f418a538b

Protocol a100-2026-10-24, text wiki (WikiText-2 test):

Protocol a100-2026-10-24, text chat (OpenAssistant validation conversations (English threads as User/Assistant text)):

Memory

Peak memory of llama-server with the command above (--port 8080 -c 8192 --jinja) answering 10 short chat prompts, llama.cpp b11617 on an NVIDIA A100 80 GB: peak GPU memory 8.4 GiB, and peak process memory (RSS) 6.5 to 6.7 GiB in two runs, which counts the parts of the memory-mapped model file that were read. The command keeps llama-server's default 8 GiB prompt cache, so system memory can grow in longer sessions; add --cache-ram 0 to limit it. Context length and other runtime settings also change memory use.

On an NVIDIA RTX 5090 with llama.cpp b11889 (-ngl 99 -c 8192 -np 1 --cache-ram 0), GPU memory in use was 7.54 GiB for the Q4_0 file alone and 7.91 GiB with the MTP drafter after the 30 speed prompts, and 7.97 GiB with mmproj-F16.gguf after the 20 chart questions.

File names, sizes and SHA-256 checksums

File name

The file name ends in a standard llama.cpp quant label, so llama.cpp, Ollama and LM Studio can pick it by tag. The file uses the standard Q4_0 label and averages 4.51 bits per weight over the whole file.

File checksums

Artemis

Teams use Artemis to test and improve code against their own objectives. See Artemis.

About and licence

Gemma 4 12B is Google's mid-sized Gemma 4 instruction model for chat, reasoning and tool use. This file is made from Google's quantisation-aware (QAT) checkpoint (google/gemma-4-12B-it-qat-q4_0-unquantized), which Google trained to hold up well at low precision.

This is an independent TurinTech conversion, not affiliated with or endorsed by Google. The file is standard GGUF and runs on stock llama.cpp. Changes from the original: the weights are converted to GGUF and quantized for llama.cpp; the vision and audio input layers are converted to a separate projector file, and Google's drafter model (gemma-4-12B-it-qat-q4_0-unquantized-assistant) to the MTP file. See NOTICE.

Apache-2.0, the licence Google releases Gemma 4 under (Gemma 4 licence page); see LICENSE and NOTICE.

When serving this file to others, for example through an API, an inference provider or OpenRouter, we'd appreciate a mention of TurinTech (turintech.ai) as the source of the quantization.

These files keep improving.