Spark-X2.5-4B GGUF

This repository contains GGUF format quants for XHToken/Spark-X2.5-4B.

Spark-X2.5-4B is a compact, high-efficiency language model featuring a hybrid attention architecture (combining sliding-window attention with full attention layers) and native context lengths up to 1M tokens.

These GGUF files were converted from the original safetensors weights using a custom Spark 2.5 build of llama.cpp at BF16 precision, then quantized into standard k-quants.

Available Files and Quantizations

File	Quant Type	Size	Description / Recommendation
`Spark-X2.5-4B-BF16.gguf`	BF16	7.66 GB	Full precision base conversion. Highest fidelity, largest size.
`Spark-X2.5-4B-Q8_0.gguf`	Q8_0	4.07 GB	Near-lossless 8-bit quantization. Recommended if you have 6GB+ VRAM.
`Spark-X2.5-4B-Q6_K.gguf`	Q6_K	3.15 GB	Excellent quality retention with minimal loss. Very balanced choice.
`Spark-X2.5-4B-Q5_K_M.gguf`	Q5_K_M	2.77 GB	Good balance between memory footprint and output quality.
`Spark-X2.5-4B-Q4_K_M.gguf`	Q4_K_M	2.42 GB	Fast, lightweight 4-bit quant. Great default for laptops and mobile devices.
`Spark-X2.5-4B-Q3_K_M.gguf`	Q3_K_M	2.02 GB	Lower memory footprint when RAM or VRAM is tight.
`Spark-X2.5-4B-Q2_K.gguf`	Q2_K	1.66 GB	Maximum compression. Noticeable quality degradation, best for ultra-constrained environments.

Requirements

Spark-X2.5 uses the Spark2_5ForCausalLM (spark2_5) architecture. To run these files, ensure your local llama.cpp installation or downstream GUI client includes Spark 2.5 runtime support.

How to Use

1. With llama.cpp

Run the model directly from the command line using llama-cli:

llama-cli -m ./Spark-X2.5-4B-Q4_K_M.gguf \
    -p "You are a helpful and concise assistant.\nUser: Hello! What can you do?\nAssistant:" \
    -n 512 \
    -c 4096 \
    --temp 0.7

To run as a local server:

llama-server -m ./Spark-X2.5-4B-Q4_K_M.gguf \
    --host 127.0.0.1 \
    --port 8080 \
    -c 4096

2. With Ollama

Create a Modelfile in the same directory:

FROM ./Spark-X2.5-4B-Q4_K_M.gguf

TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.7

Then build and run the model:

ollama create spark-x2.5-4b -f Modelfile
ollama run spark-x2.5-4b

3. With LM Studio

Download or copy any .gguf file to your LM Studio models folder.
Search for Spark-X2.5-4B in your local models tab.
Load the model with default GPU offloading settings and chat.

Original Model Details

Original weights: XHToken/Spark-X2.5-4B
Base Architecture: Spark 2.5 (Spark2_5ForCausalLM)
Context Length: Up to 1,048,576 tokens (1M)
Vocabulary Size: 131,072 tokens
License: Apache 2.0

Attribution & Credits

Original model weights and architecture developed by XHToken.

GGUF conversions produced by Pink Pixel.

Made with 💖 by Pink Pixel

sizzlebop/Spark-X2.5-4B-GGUF