KVARK Sparkle 1.0 (27B)

KVARK Sparkle 1.0 answers in about half the time of Qwen3.8-27B at the same accuracy.

The demo runs the GGUF version (Q4_K_M) with llama.cpp.

Variants

Weights	Repository	Size
BF16, with MTP head	this repository	52 GB
FP8, with MTP head, for vLLM	Kvark-Sparkle-1.0-27B-FP8	29 GB
GGUF Q8_0 to IQ4_XS, with MTP head, for llama.cpp	Kvark-Sparkle-1.0-27B-GGUF	15.3 to 29.0 GB

Model details

Architecture	Qwen3.8-27B, unchanged
Parameters	27.8B, BF16
Layers	64: 48 linear attention, 16 full attention
Full attention	24 query heads, 4 key-value heads, head dim 256
Hidden size, vocabulary	5,120, 248,320 tokens
Context	262,144 tokens native; evaluated up to 64k
KV cache	64 KiB per token in BF16, 32 KiB in FP8. Only the 16 full-attention layers store it.
MTP	1 layer, carried over from the base model, used for speculative decoding
Vision encoder	27 layers

Results

Both models ran with FP8 weights, thinking on, a 64k token cap and the same prompts. Sampling was temperature 1.0, top_p 0.95 and top_k 20. Differences are paired over items, with 95% bootstrap intervals.

Benchmark	Accuracy (%)			Median reasoning tokens			64k cap hits
Benchmark	Sparkle	Qwen	Difference	Sparkle	Qwen	Cut	64k cap hits
GPQA Diamond198 items × 5 seeds	89.3	89.4	−0.1[−1.9, +1.8]	3,122	7,854	−60%	9 / 34
AIME2025 and 2026 · 60 problems × 5 seeds	93.3	94.0	−0.7[−3.3, +2.0]	5,502	14,318	−62%	12 / 12
LiveCodeBenchv6 · 50 recent problems × 3 seeds	82.7	77.3	+5.3[0.0, +11.3]	6,664	19,950	−67%	15 / 27
IFEval541 prompts, strict	91.3	92.4	−1.1[−2.4, +0.2]	470	714	−34%	0 / 0
MMLU-Pro980 items, 70 per category	83.3	84.0	−0.7[−2.2, +0.8]	429	727	−41%	0 / 5

Speed

One user, 1 × H200	Sparkle, MTP	Qwen, MTP	Qwen, no MTP
Median time per answer	13.7 s	28.1 s	49.1 s
MTP mean acceptance length	2.53	2.50	off

Measured with vLLM 0.30 and FP8 weights on 20 fixed questions (10 GPQA Diamond, 10 LiveCodeBench), one request at a time, thinking on.

Quickstart

vLLM (FP8 with MTP)

vllm serve Egzakta/Kvark-Sparkle-1.0-27B-FP8 \
  --max-model-len 131072 --kv-cache-dtype fp8 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="none")
response = client.chat.completions.create(
    model="Egzakta/Kvark-Sparkle-1.0-27B-FP8",
    messages=[{"role": "user", "content": "How many primes are there between 100 and 150?"}],
    max_tokens=65536,
)
message = response.choices[0].message
print(getattr(message, "reasoning", None))
print(message.content)
print(response.usage.completion_tokens, "tokens")

llama.cpp

llama-server -hf Egzakta/Kvark-Sparkle-1.0-27B-GGUF:Q4_K_M -c 65536 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 3

Transformers (BF16)

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "Egzakta/Kvark-Sparkle-1.0-27B"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [{"role": "user", "content": [{"type": "text", "text": "How many primes are there between 100 and 150?"}]}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
output = model.generate(**inputs, max_new_tokens=65536)
print(processor.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Thinking and sampling

Thinking is on by default. To turn it off for a request, pass extra_body={"chat_template_kwargs": {"enable_thinking": False}}. Use temperature 1.0, top_p 0.95 and top_k 20, the defaults in generation_config.json. For hard math and code, allow max_tokens of 65,536, the cap used in our evaluation.

How it was trained

Sparkle 1.0 merges two fine-tuned lines of Qwen3.8-27B. Both were trained with LoRA on the language modules only, so the vision encoder is the base model's.

Efficiency line. GRPO on the base model with rewards for accuracy and against overlong thinking, then SFT on the shortest correct solutions sampled from that checkpoint, then a short accuracy-first GRPO stage with a length penalty.
Knowledge line. SFT with a fresh LoRA on the base model. Math keeps the median-length correct solution for hard problems (2 to 6 of 8 samples correct) and the shortest for easy ones. Science keeps median-length correct solutions to SuperGPQA physics, chemistry and biology questions. Code keeps the shortest correct solution. Instruction following uses 1,249 prompts with verifiable constraints, with the IFEval test prompts excluded, and keeps the median-length response that passes a strict checker.
Weight averaging: base + 0.5 × (efficiency − base) + 0.5 × (knowledge − base).
Ten steps of sequence-level GSPO on the averaged model, with no KL term and a 16k cap.

Training prompts come from DeepMath, LiveCodeBench v1 to v5, SuperGPQA and IF_multi_constraints. Responses were sampled from Qwen3.8-27B and our own checkpoints and checked by verifiers. The one exception is 27 of the 1,338 SFT examples in the efficiency line: solutions from Qwen3.8-Flash-Next to problems our checkpoint could not solve. No outputs of proprietary models were used.

License

Apache 2.0, inherited from Qwen3.8-27B. See LICENSE and NOTICE. Commercial use is permitted, with no revenue cap.

Contact us

hello@kvark.ai

Citation

@misc{kvarksparkle2026,
  title  = {KVARK Sparkle 1.0},
  author = {Egzakta AI Lab},
  year   = {2026},
  url    = {https://huggingface.co/Egzakta/Kvark-Sparkle-1.0-27B}
}