Muse-Glimmer-30B-lecore-GGUF
GGUF builds of meta-models/Muse-Glimmer-30B for llama.cpp, Ollama and LM Studio. These are companions to the MLX export staccs/Muse-Glimmer-30B-lecore-mlx, which has the measured perplexity numbers.
| File | Size | What it is |
|---|---|---|
Muse-Glimmer-30B-Q4_K_M.gguf |
16.94 GB | stock llama.cpp Q4_K_M |
Muse-Glimmer-30B-lecore-mix.gguf |
15.84 GB | leCore per-tensor mix: each linear layer gets the GGUF K-quant matching the bits leCore measured for it (2→Q2_K, 3→Q3_K, 4→Q4_K, 8→Q8_0); embeddings and output follow the MLX export |
Muse-Glimmer-30B-mmproj-f16.gguf |
3.85 GB | stock llama.cpp Q4_K_M |
The lecore-mix file is quantized from the original bf16 weights with llama.cpp K-quants. The bit width of each tensor follows leCore's measured (KL-damage) allocation, so its numbers are close to, but not identical to, the MLX export.
Some tensor widths are not multiples of 256, so llama.cpp may fall back to a compatible type for those tensors.
Usage
Ollama: download a file, then
printf 'FROM ./Muse-Glimmer-30B-Q4_K_M.gguf
' > Modelfile && ollama create muse-glimmer-30b-lecore-gguf -f Modelfile && ollama run muse-glimmer-30b-lecore-gguf
or pull straight from the Hub with ollama run hf.co/staccs/Muse-Glimmer-30B-lecore-GGUF:Q4_K_M
LM Studio: search for staccs/Muse-Glimmer-30B-lecore-GGUF in the model browser and pick a file. On a 36 GB Mac, the lecore-mix file is the one that fits.
llama.cpp
llama-cli -hf staccs/Muse-Glimmer-30B-lecore-GGUF -p "Hello" -n 256
Benchmarks (lm-eval-harness 0.4.9, run on the shipped weights dequantized to bf16 on GPU)
MMLU 5-shot on the first 10 questions of each of the 57 subjects; GSM8K 5-shot on the first 100 test problems (greedy, 320 new tokens, no chat template). These are subsets, so expect about ±2–4 points of sampling noise.
| Model | MMLU (5-shot) | GSM8K strict | GSM8K flexible |
|---|---|---|---|
| leCore mix (this repo) | 81.8% | 96.0% | 96.0% |
| stock 3-bit affine, group 64 (simulated) | 78.6% | 90.0% | 90.0% |
License
Base model: meta-models/Muse-Glimmer-30B, released under the Apache License 2.0. Only quantization was done here, with no retraining. Vision: this MLX export is text-only, because mlx-lm's Muse-Glimmer port has no vision tower. The companion GGUF repo includes an unquantized f16 mmproj vision projector for llama.cpp image input, if its conversion succeeded. Perplexity and benchmarks below are text-only.
Related builds
Same base model, other formats in the lecore set:
| repo | format | size |
|---|---|---|
| Muse-Glimmer-30B-lecore-mlx | MLX (Apple Silicon) | 16.04 GB |