Sparset/Thinklite-27B-GGUF

[!NOTE] Thinklite-27B in a 16.5 GB Q4_K_M that keeps its efficiency. At reasoning_effort="xhigh", this Q4 file uses 48.4% less reasoning than Qwen3.8-27B quantized the same way and keeps 99.3% of its average accuracy (90.9 against 91.5), so a complete answer arrives about 1.9x sooner for one user. At reasoning_effort="low" it uses 24.8% less reasoning and keeps 96.7% of the base model's accuracy.

Check out the full-precision Thinklite-27B, which cuts 47% of reasoning tokens at 99.6% of Qwen3.8-27B's accuracy.

• Both files were quantized here with the identical recipe and run with the same llama.cpp build, settings and seeds, on 200 GSM8K, 30 AIME 2025, 150 MMLU-Pro and 200 IFEval questions sampled from our frozen benchmark suite, plus all 131 LiveCodeBench v6 problems and all 1,240 BFCL v4 core cases. Every number compares Thinklite-27B Q4_K_M with Qwen3.8-27B Q4_K_M.
• Reasoning vs base: change in mean reasoning tokens per answer against the base model, averaged over the six benchmarks. Quality retained: Thinklite's average accuracy as a share of the base model's. TTFO speed-up (time-to-final-output): how much sooner one user gets the complete answer, thinking included. Per benchmark it is the base model's total output tokens divided by Thinklite's, scaled by the measured single-user speed; the table shows the average over the six benchmarks. On this GPU both Q4 files decode at the same speed (69.46 vs 70.09 tokens/s for one user).

Side-by-side comparisons (full-precision Thinklite-27B)

These two videos come from the full-precision (BF16) Thinklite-27B served with vLLM, not from this Q4 file. They show what the model does; the tables on this page show what the Q4 file keeps of it.

Debugging

Full-precision Qwen3.8-27B and Thinklite-27B get the same debugging prompt (retry code plus two logs: find the duplicate charges to refund and the fix) with the same seed, each model on one NVIDIA RTX PRO 6000 serving 20 requests at once. Both answer correctly; Thinklite-27B is done in 2:36.9 with 3,381 tokens, the base model in 11:04.9 with 14,313 tokens (4.2x). The video plays at 22x speed and the clocks show the measured time. This is the largest speed-up of 20 paired runs of this prompt. Thinklite-27B was correct in 19 of the 20 runs and the base model in all 20; over the 19 correct runs the median speed-up was 2.7x.

Building a 3D physics lab

Both full-precision models get the same objective: build an interactive 3D science lab as a single HTML file in which everything has to follow realistic physics. Objects have real sizes and masses and fall under real gravity (9.81 m/s²): a hinged door swings with its momentum and stops at its frame, glassware tips over when bumped and shatters when it hits something hard, a rubber ball bounces high while a heavy steel ball barely bounces, a block tower topples and a pendulum swings. Both models run at reasoning_effort="xhigh" on the same NVIDIA RTX PRO 6000. The numbers are the total tokens and generation time each model needed to reach a working version of the lab: Thinklite-27B 87,134 tokens in 56.7 min, the base model 127,661 tokens in 87.6 min (1.5x). The video shows one working game per model; it is not a measure of how often each model succeeds.

How it works

Reasoning models spend much of their output on thinking that does not change the answer: restating the question, repeating a step, re-checking a result that was already checked. Thinklite-27B is fine-tuned from Qwen3.8-27B on its own verified-correct answers to reach the same answers by a shorter route. Only the thinking gets shorter; the visible answer keeps its full length and format.

This repository is that model quantized to 4 bits for llama.cpp, Ollama and LM Studio. The quantization uses an importance matrix: before rounding, the full-precision model is run over a broad calibration text, and the rounding then protects the weights whose inputs matter most. The file is a standard Q4_K_M; only the choice of rounded values differs.

Benchmarks

Setup:

• Both Q4_K_M files were made here with the identical recipe: llama.cpp v0.8.0 (commit 8642541b255b), BF16 GGUF from the original weights, the same importance matrix calibration, then llama-quantize --imatrix to Q4_K_M. Qwen3.8-27B: revision 1d4bf0f2ff60; Thinklite-27B: revision 52b8fbe1f025. For LiveCodeBench and BFCL the Qwen3.8-27B Q4 file was rebuilt with the same recipe on another GPU. It is not byte-identical to the file used for the other four benchmarks (its importance matrix differs in the last bits) but sits at the same distance from BF16 (mean KL divergence 0.0143 against 0.0148). Thinklite's Q4 file is this repository's file in both runs.
• Served with llama-server, q8_0 KV cache, 16 parallel requests, temperature 1.0, top_p 0.95, top_k 20, the same seed per question for both models.
• 200 GSM8K, 30 AIME 2025, 150 MMLU-Pro and 200 IFEval questions sampled from our frozen benchmark suite, plus all 131 LiveCodeBench v6 problems and all 1,240 BFCL v4 core cases, one answer per question, scored with the suite's checkers (math-verify, option letter, lm-eval IFEval prompt-level strict, the official LiveCodeBench evaluator on public and private tests, the official BFCL AST checker). Answers that hit their cap (32,768 or 49,152 tokens, 65,536 for LiveCodeBench) were rerun once with a 131,072-token cap; truncated or unparseable answers count as incorrect.

reasoning_effort="xhigh"

Mean reasoning tokens, paired differences and invalid answers (Q4 files)

reasoning_effort="low"

Mean reasoning tokens, paired differences and invalid answers (Q4 files)

Usage

llama.cpp

llama-server -hf Sparset/Thinklite-27B-GGUF:Q4_K_M --jinja --reasoning-format deepseek -c 65536

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8080/v1", api_key="-")
reply = client.chat.completions.create(
    model="thinklite",
    messages=[{"role": "user", "content": "How many primes are there below 100?"}],
    temperature=1.0, top_p=0.95,
    extra_body={"top_k": 20, "chat_template_kwargs": {"enable_thinking": True, "reasoning_effort": "xhigh"}},
)
print(reply.choices[0].message.content)

--jinja uses the model's own chat template, which is what reads reasoning_effort ("low", "medium" or "xhigh", exactly as with Qwen3.8-27B). --reasoning-format deepseek returns the thinking separately from the answer. Long xhigh answers on hard problems can exceed 32k tokens, so give each request enough context.

Ollama

ollama run hf.co/Sparset/Thinklite-27B-GGUF:Q4_K_M

Images: this file is the language model only. Thinklite-27B's vision encoder is byte-for-byte Qwen3.8-27B's, so the vision projector (mmproj) of a Qwen3.8-27B GGUF works with it, for example --mmproj with the projector from ggml-org/Qwen3.8-27B-GGUF.

Files

SHA-256 checksums

Limitations

• GSM8K, AIME 2025, MMLU-Pro and IFEval are samples of their benchmarks, one answer per question; AIME 2025 has 30 questions, so one answer moves it by 3.3 points. The 95% intervals in the details above show which differences are within noise.
• Thinklite-27B Q4 is below Qwen3.8-27B Q4 where the tables show it, most clearly on AIME 2025 and, at low effort, LiveCodeBench. See the full-precision card for the full-precision model, measured with vLLM.
• No vision encoder or multi-token-prediction head in this file; use a Qwen3.8-27B projector for images (see Usage).
• No safety tuning beyond the base model's. Check the model on your own tasks before relying on it.

License

Apache-2.0, the license of the base model Qwen/Qwen3.8-27B.