Spark-X2.6-4B-abliterated-GGUF

By Vatsal Bajpai (@xblack) at FrostWolfAI.

_{Everything here is released openly. If it's useful to you and you'd like to support more open-source work on AI, a coffee goes a long way — entirely optional.}

GGUF quantizations of FrostWolfAI/Spark-X2.6-4B-abliterated, a decensored variant of XHToken/Spark-X2.6-4B produced with Heretic v2 using Arbitrary-Rank Ablation (ARA).

Refusals drop from 100/100 to 2/100 at a KL divergence of 0.0177. Architecture, 1M-token context, controllable thinking and the <tool_call> format are all intact. Full details, abliteration parameters and the reproduction bundle live on the safetensors model card.

Who this is for: local agents, coding and security research with llama.cpp on a consumer GPU, a laptop or CPU-only. At 4-bit the weights are about 2.4-2.6 GB.

Files

Full quantization set (BF16 + 14 quants, 44.53 GB total), converted from the BF16 safetensors with upstream llama.cpp convert_hf_to_gguf.py (master, 2026-10-29) and quantized with llama-quantize (build 11669, 620d2225c). No importance matrix was used.

[!IMPORTANT] These files use the spark2_5 GGUF architecture. You need a llama.cpp build with Spark2.5 support (tested with build 11669); older builds will fail to load the model. Apps that bundle llama.cpp (Ollama, LM Studio, Jan, ...) work once their bundled engine includes spark2_5.

Quickstart

# llama.cpp - downloads the quant straight from this repo
llama-server -hf FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF:Q4_K_M --jinja -ngl 99 -fa on -c 32768

# OpenAI-compatible request; thinking is ON by default
curl -s http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "Explain how a TLS 1.3 handshake works."}],
  "temperature": 1.0, "top_p": 0.95, "top_k": 0,
  "chat_template_kwargs": {"enable_thinking": false}
}'

# CLI chat
llama-cli -hf FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF:Q6_K --jinja -ngl 99 -cnv

Always pass --jinja: the embedded chat template handles thinking control and tool calls.

Generation parameters

XHToken's recommended settings: temperature=1.0, top_p=0.95, top_k=0 (disabled), repetition_penalty=1.0.

Thinking

Spark-X2.6 thinks by default; llama-server returns the reasoning in reasoning_content and the answer in content. Control it per request with chat_template_kwargs:

• {"enable_thinking": false} - no reasoning, direct answer
• {"reasoning_effort": "low"} / "high" (default) - reasoning budget

With thinking on, give it room: short max_tokens budgets can be spent entirely inside <think>.

Tool use

Send OpenAI-style tools with --jinja; llama-server parses Spark's native <tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call> output into standard tool_calls with JSON arguments, ready for agent frameworks.

Tested

Each quant below was served with llama-server --jinja -ngl 99 -fa on -c 8192 and checked for a direct answer (thinking off), a reasoned answer (thinking on: 17 x 23 = 391) and a structured tool call (get_weather(city="Paris"), finish_reason=tool_calls):

Speed

llama-bench (-ngl 99 -fa 1 -p 512 -n 128, 3 reps) on an RTX 4090 power-limited to 360 W:

Runs anywhere

Weights only. Three of every four layers use a 512-token sliding window, so the KV cache grows far more slowly than in a pure global-attention 4B; at the full 1M-token window it is still a server workload.

Model details

Responsible use

Refusal suppression is deliberate and works as intended: this model will comply with requests the base model would refuse, including some it should not. There is no safety filtering layered on top. You are responsible for how you deploy it - don't put it behind an unmoderated public-facing endpoint serving third parties. It inherits Spark-X2.6-4B's factual limitations; abliteration removes refusal directions, it does not add capability or judgment.

Made by FrostWolfAI - the uncensored base behind the upcoming Frost Cyber Mini 1.1.

License

Apache-2.0, inherited from XHToken/Spark-X2.6-4B.

Related

• FrostWolfAI/Spark-X2.6-4B-abliterated - BF16 safetensors, abliteration details, reproduction bundle
• XHToken/Spark-X2.6-4B - the original post-trained model
• p-e-w/heretic - the abliteration tool
• ggml-org/llama.cpp - inference engine