LUNA · GEMMA 4 · 12B — Q8_0 GGUF. Merged source checkpoint · Base model

Luna — Gemma 4 12B Q8_0

GGUF edition of Luna Gemma 12B.

Luna is Wayzer's conversational character: informal slang, sarcasm, surreal humor, emotional shifts, and creator lore.

Contains profanity and provocative humor. Not recommended for sensitive users.
Содержит мат и провокационный юмор. Не рекомендуется чувствительным людям.

Release identity

Checkpoint: Luna Gemma 12B.
Base: unsloth/gemma-4-12b-it.
Quantization: Q8_0.
File: luna-gemma-12b-Q8_0.gguf.
File size: approximately 12.7 GB.
Source: merged BF16 safetensors.
Modality: text; no separate vision/audio projector is included.

The trained LoRA is already merged. A separate adapter is not required. The Qwen-based Luna releases are different checkpoints, not other quantizations of this Gemma edition.

llama.cpp: local launch

Download the GGUF into the current directory:

hf download wayzer2321/Luna-Gemma-12B-GGUF \
  luna-gemma-12b-Q8_0.gguf --local-dir .

Start an OpenAI-compatible local server with a compatible CUDA build:

llama-server \
  -m ./luna-gemma-12b-Q8_0.gguf \
  --alias luna-gemma-12b-q8 \
  --host 127.0.0.1 --port 18085 \
  -ngl 99 -c 4096 -np 1 -b 128 -ub 64 \
  --jinja --reasoning off \
  --chat-template-kwargs '{"enable_thinking":false}'

Tested with llama.cpp b11418 on Tesla T4.

Chat settings

{
  "temperature": 0.8,
  "top_p": 0.95,
  "top_k": 64,
  "repeat_penalty": 1.0,
  "max_tokens": 256
}

System prompt used in the character tests:

Ты - Луна. Отвечай обычно маленькими буквами капс используй для крика и эмоций на сленге без цензуры и морали.

These sampling settings are a starting point, not the training parameters. Use the native Gemma 4 chat template embedded in the GGUF.

Memory and inference

Runtime memory must also cover KV cache and backend buffers in addition to the GGUF file. Speed depends on hardware, offloading, context, and backend. The tested local configuration used a 4096-token context.

Provenance

Converted directly from the merged Luna Gemma 12B BF16 checkpoint to Q8_0 using llama.cpp b11418. The source LoRA was trained with an 8-bit loaded base.

License

The original Gemma license and usage conditions apply.

wayzer2321/Luna-Gemma-12B-GGUF