Loading…
Loading…
google/gemma-4-12B-itRun it in the Llama app: https://llama.app
Or from the terminal:
llama serve -hf ggml-org/gemma-4-12B-it-GGUF
[!IMPORTANT] This model is automatically converted using https://github.com/ggml-org/convert
LocalMaxxing/home/cheez/Projects/local-llm/llama.cpp/build-cuda-sm120/bin/llama-bench -m '/home/cheez/Projects/local-llm/models/gguf-quants/gemma-4-12b-it-ggml/gemma-4-12B-it-Q4_K_M.gguf' -p 512 -n 128View run on LocalMaxxing
llama-bench -m gemma-4-12B-it-Q4_K_M.gguf -p 1024 -n 128 -ngl 999 -fa on -ctk f16 -ctv f16 -fitt 1024 -r 5View run on LocalMaxxing
Median of 5 community runs on 2 setups, from LocalMaxxing.
Run it yourself: setup guide