Spark-X2.6-4B-GMCE (GMCoder Efficiency)
XHToken/Spark-X2.6-4B fine-tuned for GMCoder Efficiency, which cuts how many tokens the model spends reasoning before it answers. Files: GGUF (Q8_0, Q6_K, Q5_K_M, Q4_K_M, BF16) and BF16 safetensors.
GGUF needs llama.cpp release b11454 or newer (spark2_5 support). Run llama-server with --jinja so the chat template's enable_thinking / reasoning_effort options work. The safetensors use the base model's remote code, so load them with trust_remote_code=True.
What it does (measured)
Single run per model, one grader (blind to model identity), approximate scores, so differences of about 5 points are noise. Our own unseen "Form B" exams, 60k token cap, both models at their default setting (the base template defaults to high reasoning effort).
| Ultimate (/25) | Historical (/100, approx) | Historical tokens | Total tokens, 3 exams | |
|---|---|---|---|---|
| Spark-X2.6-4B base, default | 0 (hit 60k cap, no answer) | ~84.5 | 35.9k | 155.9k |
| Spark-X2.6-4B-GMCE | 19.5 (finished at 43.1k) | ~70.5 | 5.9k (-84%) | 109.0k (-30%) |
- On Ultimate (25 math/logic questions) the base thought until the cap and never wrote an answer; GMCE's shorter reasoning finished and scored 19.5/25.
- On Historical, GMCE used 84% fewer tokens but scored about 14 points lower.
- On Everyday, both the base and GMCE ran to the 60k cap without answering. This model family tends to keep re-drafting on that exam in thinking mode, and GMCE does not fix it.
Caveats
- This is a trade: much less reasoning for some accuracy on the exam where the base did finish.
- The base with thinking turned off (
enable_thinking=false) also uses few tokens (about 7k on Historical and Everyday), but it still never answered Ultimate within the cap. - Benchmarks are our own and small, one run each. Do not read the numbers as leaderboard results.
Training
QLoRA (NF4) with LoRA r8 / alpha 8 on q_k_v_proj, out_proj, g_proj, gate_proj, up_proj, down_proj (16.2M trainable params), LR 5e-5, batch 1, 16 epochs, completion-only loss, on the 531-example GMCoder corpus. The adapter was merged into the original BF16 base weights.