Spark-X2.5-4B GGUF
GGUF quantized versions of XHToken/Spark-X2.5-4B, a reasoning-capable language model designed for efficient deployment.
Model Overview
Spark-X2.5-4B is a high-performance 4B parameter model specializing in reasoning and text generation. It supports a thinking/reasoning mode where the model generates a chain-of-thought process before providing the final answer, allowing for more accurate and complex problem solving.
The model utilizes a structured conversation format with role-based delimiters (<|System|>, <|User|>, <|Bot|>) and explicitly handles reasoning via <think> blocks.
Model Architecture
| Property | Value |
|---|---|
| Architecture | Transformer |
| Parameters | 4B |
| Supported Languages | en, zh |
Available GGUF Files
| File | Quantization | Use Case |
|---|---|---|
spark-x2.5-4b-BF16.gguf |
BF16/FP16 | Full precision |
spark-x2.5-4b-Q4_0.gguf |
Q4_0 | Standard 4-bit quantization |
spark-x2.5-4b-Q4_K_S.gguf |
Q4_K_S | Small 4-bit K-quant |
spark-x2.5-4b-Q4_K_M.gguf |
Q4_K_M | Medium 4-bit K-quant (Recommended) |
spark-x2.5-4b-Q5_K_S.gguf |
Q5_K_S | Small 5-bit K-quant |
spark-x2.5-4b-Q5_K_M.gguf |
Q5_K_M | Medium 5-bit K-quant |
spark-x2.5-4b-Q6_K.gguf |
Q6_K | 6-bit K-quant |
spark-x2.5-4b-Q8_0.gguf |
Q8_0 | 8-bit quantization |
Usage
llama.cpp CLI
./llama-cli \
-m spark-x2.5-4b-Q4_K_M.gguf \
-p "Explain quantum computing in simple terms." \
--temp 1.0 --top-p 0.95 --top-k 20
llama-server (OpenAI-compatible API)
./llama-server \
-m spark-x2.5-4b-Q4_K_M.gguf \
--host [IP_ADDRESS] --port 8080
Thinking Mode
Thinking mode is integrated into the model's output. The model generates its internal reasoning process within <think> and </think> tags before providing the final answer. This behavior can be controlled via the enable_thinking parameter in the chat template.
Tool Calling
The model supports structured tool calling. Tool definitions are provided in the system prompt, and the model responds using <tool_call> blocks with specific argument tags, which are then processed and returned via tool response tags.
Generation Parameters
Recommended parameters for optimal performance:
| Parameter | Value |
|---|---|
| Temperature | 1.0 |
| Top-P | 0.95 |
| Top-K | 20 |
Quantization
These GGUF files were created from the source model using llama-quantize from the llama.cpp project.
Acknowledgements
- Original model: XHToken/Spark-X2.5-4B
- Quantization tool: llama.cpp