KVARK Sparkle 1.0 (27B) GGUF
GGUF files of KVARK Sparkle 1.0 for llama.cpp, Ollama and LM Studio. Every file keeps the MTP head. The quants use an importance matrix computed on our own SFT data (imatrix.gguf, included).
The demo runs Kvark-Sparkle-1.0-27B-Q4_K_M.gguf from this repo with llama.cpp.
Files
| File | Quantization | Size |
|---|---|---|
| Kvark-Sparkle-1.0-27B-Q8_0.gguf | 8.5 bits per weight | 29.0 GB |
| Kvark-Sparkle-1.0-27B-Q6_K.gguf | 6.56 bits per weight | 22.4 GB |
| Kvark-Sparkle-1.0-27B-Q5_K_M.gguf | 5.5 bits, some tensors at 6.56 | 19.5 GB |
| Kvark-Sparkle-1.0-27B-Q4_K_M.gguf | 4.5 bits, some tensors at 6.56recommended for 24 GB cards | 16.8 GB |
| Kvark-Sparkle-1.0-27B-IQ4_XS.gguf | 4.25 bits per weight | 15.3 GB |
| mmproj-Kvark-Sparkle-1.0-27B-F16.gguf | vision projector, F16 | 0.9 GB |
Memory
| Context | KV cache, f16 (default) | KV cache, q8_0 |
|---|---|---|
| 32,768 | 2 GiB | 1.1 GiB |
| 65,536 | 4 GiB | 2.1 GiB |
| 131,072 | 8 GiB | 4.3 GiB |
| 262,144 | 16 GiB | 8.5 GiB |
| Linear-attention state, any context | under 0.2 GB | under 0.2 GB |
Run with llama.cpp
llama-server -hf Egzakta/Kvark-Sparkle-1.0-27B-GGUF:Q4_K_M -c 65536 --jinja \
--spec-type draft-mtp --spec-draft-n-max 3 \
--temp 1.0 --top-p 0.95 --top-k 20
| Flag | Effect |
|---|---|
--spec-type draft-mtp --spec-draft-n-max 3 | speculative decoding with the included MTP head |
-m Kvark-Sparkle-1.0-27B-Q4_K_M.gguf --mmproj mmproj-Kvark-Sparkle-1.0-27B-F16.gguf | load local files |
--no-mmproj | text only, skip the vision projector |
-ctk q8_0 -ctv q8_0 | halve the KV cache; flash attention switches on by itself |
--reasoning off | answer without thinking |
The server speaks the OpenAI chat API on port 8080 and returns the thinking in reasoning_content:
curl -s localhost:8080/v1/chat/completions -H "Content-Type: application/json" \
-d '{"messages": [{"role": "user", "content": "How many primes are there between 100 and 150?"}]}'
Ollama and LM Studio
ollama run hf.co/Egzakta/Kvark-Sparkle-1.0-27B-GGUF:Q4_K_M
In LM Studio, search for Egzakta/Kvark-Sparkle-1.0-27B-GGUF. Set the context length to 65,536 tokens and the sampling to temperature 1.0, top_p 0.95 and top_k 20.
More
Training, data and the BF16 weights are on the main model card. License: Apache 2.0, commercial use permitted, no revenue cap.