[!Note] This quant ONLY works in vLLM and NOT SGLang. The
lm_headis quantized to FP8 and SGLang can't load it
Qwen3.8-27B-Unsloth-NVFP4-Sharp
This is Unsloth's Dynamic NVFP4 quantization of Qwen3.8-27B with the Qwen Sharp chat template embedded in place of the original. The weights are untouched — identical to Unsloth's release. The only things that differ are the chat template and this README.
It stacks two independent contributors to inference speed. Unsloth's NVFP4 quantization makes each token faster to generate. The Sharp template makes the model use fewer tokens to reach an answer and ships with a medium reasoning-effort default instead of xhigh. Measurements of this build against the stock release are below.
What's inside
Weights: unsloth/Qwen3.8-27B-NVFP4, commit 7d6f8d4d72f56b92b3cdbf22f156b90e1bab0108 — Unsloth's per-layer calibrated Dynamic NVFP4; their documentation covers the method.
Chat template: peculiar-ragdoll/Qwen-Sharp-Chat-Templates, commit 3dc34df52c63dd22ada21f96435e069deaa8d7da. The template file identifies itself internally as qwen3.8-froggeric-v22.1 — Sharp is built on top of froggeric's Qwen-Fixed-Chat-Templates v22.1. In the author's words, the goal is "more information per token by cutting filler without sacrificing correctness or substance."
Tested on this build
Summary. This build gives up a small amount of accuracy for a large gain in speed.
Accuracy: MMLU-Pro 78.8% against stock's 79.5% and 80.2%, inside the run-to-run spread. Tool calling 2–3 points below stock in both normal and hard mode.
Speed against stock as shipped: 2.5× fewer tokens and 2.9× less time on MMLU-Pro, a quarter faster per tool-calling turn, and 16–50% faster generation at almost every concurrency and context depth. Most of this comes from the medium effort default.
Speed against stock at medium effort: 13% fewer tokens and 8% less time on MMLU-Pro, 3–9% faster generation, but 8–9% more tokens on tool calling. Prompt processing is identical throughout.
MMLU-Pro
Three configurations on a single DGX Spark: this build at its medium default, stock at its shipped xhigh default, and stock at medium, which separates the template's effect from the effort change. 140 MMLU-Pro questions (10 per category), 3 runs per configuration at temperature 0.7, using the official TIGER-AI-Lab prompt and answer scoring, with no token or time limits. Wall clock is the sum of each question's latency. The vLLM setup is identical throughout; only the template and reasoning effort differ.
this build @ medium (default) |
stock @ medium |
stock @ xhigh (shipped) |
|
|---|---|---|---|
| accuracy (mean ± standard deviation of 3 runs) | 78.8% ± 2.2 | 79.5% ± 0.8 | 80.2% ± 0.4 |
| tokens generated per question | 1,140 | 1,307 | 2,839 |
| wall clock per question | 78 s | 84 s | 228 s |
| summed wall clock per run | 3.0 h | 3.3 h | 8.9 h |
Accuracy is the same across all three: the gaps are smaller than the run-to-run spread. The difference is cost: this build matches the stock release as shipped on 2.5× fewer tokens and 2.9× less wall clock. The template alone — identical effort, weights, and settings — cuts tokens by 13% and wall clock by 8%.
Tool calling (tool-eval-bench)
The same three configurations on tool-eval-bench: 69 tool-calling scenarios in normal mode, 84 in hard mode, each scored pass, partial, or fail. One run each at the benchmark's defaults (temperature 0, 4,096-token reply limit). Median turn is the benchmark's own timing.
this build @ medium (default) |
stock @ medium |
stock @ xhigh (shipped) |
this build vs stock medium | this build vs stock xhigh | |
|---|---|---|---|---|---|
| score, normal mode | 89 | 91 | 90 | −2 | −1 |
| score, hard mode | 86 | 88 | 89 | −2 | −3 |
| tokens generated, normal mode | 284k | 265k | 313k | +8% | −9% |
| tokens generated, hard mode | 353k | 324k | 390k | +9% | −10% |
| median turn, normal mode | 4.4 s | 5.1 s | 6.0 s | −14% | −27% |
| median turn, hard mode | 4.5 s | 5.1 s | 6.0 s | −11% | −24% |
This build scores 2–3 points below stock (66 pass / 13 partial / 5 fail in hard mode against stock's 68 / 11 / 5 and 70 / 9 / 5). It generates 8–9% more tokens than stock at the same effort and 9–10% fewer than stock as shipped, with the fastest turns of the three. Its weakest category is D Restraint & Refusal at 67%.
Throughput (llama-benchy)
Serving speed on the same hardware: 2,048-token prompts and 128-token replies, at context depths from 0 to 100,000 tokens and 1 to 10 simultaneous requests. Each figure is the mean of 3 runs.
Generation, tokens/s (no context → 100k context)
| concurrency | this build @ medium (default) |
stock @ medium |
stock @ xhigh (shipped) |
this build vs stock medium | this build vs stock xhigh |
|---|---|---|---|---|---|
| 1 | 23.7 → 20.3 | 21.7 → 19.0 | 17.1 → 17.5 | +9% → +7% | +39% → +16% |
| 2 | 41.1 → 34.3 | 38.9 → 31.6 | 29.4 → 22.8 | +5% → +8% | +40% → +50% |
| 5 | 73.5 → 41.9 | 71.5 → 39.1 | 49.0 → 31.3 | +3% → +7% | +50% → +34% |
| 10 | 99.0 → 46.4 | 107.7 → 45.8 | 99.4 → 38.6 | −8% → +1% | 0% → +20% |
Prompt processing, tokens/s (no context → 100k context)
| concurrency | this build @ medium (default) |
stock @ medium |
stock @ xhigh (shipped) |
this build vs stock medium | this build vs stock xhigh |
|---|---|---|---|---|---|
| 1 | 2,103 → 303 | 2,316 → 307 | 2,164 → 307 | −9% → −1% | −3% → −1% |
| 2 | 1,721 → 295 | 1,717 → 285 | 1,834 → 295 | 0% → +3% | −6% → 0% |
| 5 | 1,517 → 273 | 1,458 → 270 | 1,491 → 270 | +4% → +1% | +2% → +1% |
| 10 | 1,328 → 255 | 1,306 → 262 | 1,307 → 259 | +2% → −2% | +2% → −2% |
This build generates faster than stock at the same effort at every concurrency and depth except concurrency 10 at short context, where stock is 8% faster. Against stock as shipped it generates 16–50% faster everywhere except concurrency 10 at short context, where the two are equal. Prompt processing is the same for all three.
License
Apache-2.0, same as Qwen3.8-27B and Unsloth's quantization. The weights are Qwen's and Unsloth's work. The chat template is froggeric's and peculiar-ragdoll's.