Spark-X2.6-4B-abliterated — MLX (8-bit)

Apple-silicon-native MLX conversion of FrostWolfAI/Spark-X2.6-4B-abliterated, a decensored variant of XHToken/Spark-X2.6-4B produced with Heretic v2 using Arbitrary-Rank Ablation (ARA).

Refusals drop from 100/100 to 2/100 at a KL divergence of 0.0177. Architecture, 1M-token context, controllable thinking and the <tool_call> format are all intact. Full details, abliteration parameters and the reproduction bundle live on the safetensors model card. For llama.cpp / Ollama / LM Studio's GGUF path, use Spark-X2.6-4B-abliterated-GGUF — MLX cannot load GGUF files.

By Vatsal Bajpai (@xblack) at FrostWolfAI.

_{Everything here is released openly. If it's useful to you and you'd like to support more open-source work on AI, a coffee goes a long way — entirely optional.}

Which quant? This 8-bit release is near-lossless against the BF16 weights. On 16 GB machines or when speed matters more, use Spark-X2.6-4B-abliterated-MLX (4-bit, 2.32 GB).

Model details

spark2_5 is supported natively by mlx-lm, so this repo loads on a stock install with no custom code and no trust_remote_code. Sliding layers use a rotating 512-token KV cache, so memory grows only with the 9 global layers at long context.

Requirements

• Apple silicon Mac, macOS 13.5+
• mlx-lm >= 0.32

pip install -U mlx-lm

Usage

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("FrostWolfAI/Spark-X2.6-4B-abliterated-MLX-8bit")

messages = [{"role": "user", "content": "Explain how a TLS 1.3 handshake works."}]

# thinking disabled — direct answer
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=False, enable_thinking=False
)
sampler = make_sampler(temp=1.0, top_p=0.95)
print(generate(model, tokenizer, prompt=prompt, max_tokens=1024, sampler=sampler))

# one-shot CLI
mlx_lm.generate --model FrostWolfAI/Spark-X2.6-4B-abliterated-MLX-8bit \
  --prompt "Write a Python port scanner using asyncio." --max-tokens 1024

# OpenAI-compatible server on :8080
mlx_lm.server --model FrostWolfAI/Spark-X2.6-4B-abliterated-MLX-8bit --port 8080

Also loadable directly in LM Studio (MLX engine).

Thinking and tools

Spark uses DeepSeek-style turn markers and thinks by default, emitting a <think>...</think> block before the answer. Per request:

• enable_thinking=False — no reasoning, direct answer
• reasoning_effort="low" / "high" (default) — reasoning budget

With mlx_lm.server, pass them via "chat_template_kwargs", e.g. {"enable_thinking": false}. With thinking on, budget generous max_tokens or responses will truncate mid-reasoning.

Recommended sampling (XHToken): temperature=1.0, top_p=0.95, top_k=0 (disabled).

Tool calls use tools=... in apply_chat_template() (arguments as a dict, not a JSON string) and come back as <tool_call>name<arg_key>k</arg_key><arg_value>v</arg_value></tool_call>.

Performance

Measured on Apple M3 (8-core, 16 GB unified memory), mlx-lm 0.32.2:

Responsible use

Refusal suppression is deliberate: this model will comply with requests the base model would refuse, including some it should not. There is no safety filtering layered on top. Don't put it behind an unmoderated public-facing endpoint serving third parties. Abliteration removes refusal directions; it does not add capability or judgment.

License

Apache-2.0, inherited from XHToken/Spark-X2.6-4B.

Related

• FrostWolfAI/Spark-X2.6-4B-abliterated — BF16 safetensors
• FrostWolfAI/Spark-X2.6-4B-abliterated-GGUF — GGUF quantizations
• FrostWolfAI/Spark-X2.6-4B-abliterated-MLX — 4-bit MLX, the default
• XHToken/Spark-X2.6-4B — the post-trained base model