Marathon project: github.com/thebusdriver37/Marathon Matching target model: Busdriver/Qwen3.8-27B-Marathon-GGUF
A Q4_K_M DFlash2 speculative drafter tuned for what Marathon actually sends a local Qwen3.8 27B: long agent sessions
with thinking, code edits (apply_patch), shell commands and tool results. It drafts 7 tokens per round for the
target to check, so the target produces several tokens per forward pass instead of one. It does nothing on its own;
use it beside the target.
C3 replaces the earlier R32 drafter. R32 is still available at revision
ee1d06ea.
Using another harness? C3 is tuned for Marathon's request format. In a test through the Pi coding agent (6 coding tasks per drafter, tree verification on, same server), C3 was not faster than the stock drafter: tokens per round were stock 4.82, R32 4.88, C3 4.59 (C3 vs stock -4.7%, 95% interval -13.0% to +3.9%). Pi sessions there were dominated by long uncapped thinking, where no drafter gains much. With harnesses other than Marathon, the stock
incoai/Qwen3.8-27B-DFlash2drafter may be as fast or faster; it is worth trying both. Output is the same with any drafter: the target model checks and chooses every token.
| File | Quantization | Bytes | SHA-256 |
|---|---|---|---|
Qwen3.8-27B-DFlash2-Marathon-C3-Q4_K_M.gguf |
Q4_K_M | 1,143,006,752 | af42cdeaac664eaa8e1c85e7b1f5f5e264cf1b078b42c6b63ee11eda4ffe443b |
36 tensors differ from the stock Q4_K_M drafter (the feature projection and the attention and MLP weights of all five layers); every other tensor, shape, quantization type and metadata field is unchanged.
incoai/Qwen3.8-27B-DFlash2, quantized to Q4_K_M
once after training. (A version trained on top of an already-quantized drafter lost most of its gain when it was
quantized again.)All on RTX 3090s at 250 W, scored on six held-out sessions (99K tokens; C, TypeScript, shell, Python, prose and JavaScript tasks) that no drafter was trained or selected on.
| vs R32 | |
|---|---|
| C3 alone, tokens per round (offline server-order replay) | +2.1% |
| C3 by kind of text: code edits / shell commands / thinking | +7.5% / +8% / unchanged |
| C3 with 7-node tree verification (offline replay) | +13.6% |
| C3 + tree in the live server vs R32 production, decode tok/s | about +12% (104.0 vs 93.0 tok/s) |
The live comparison is approximate: the two arms ran on different (identical-model) GPUs and, as with any sampled generation, wrote different text. Measured exactly on identical text, the tree alone adds +5.1% tokens per round at no extra cost per round (42.1 vs 43.0 ms). Thinking text is still the hard part for every drafter tested.
Marathon's runtime turns the drafter's candidate lattice into a best-first tree of 7 guesses (scores divided by a
temperature of 2.0, since the drafter is overconfident) instead of a single chain of 7, and checks the whole tree in
the same 8-row target batch the chain uses, so a round costs the same. The target still samples every token itself,
so the output distribution is unchanged. It ships with Marathon as one patch on llama.cpp v1.0.0,
patches/llama.cpp/qwen38/008-tree-verify.patch,
applied with Marathon's other Qwen runtime patches by marathon setup-llama qwen38 and switched on with
LLAMA_SPEC_TREE_NODES=7 LLAMA_SPEC_TREE_TEMP=2.0. Stock llama.cpp runs this drafter in the normal chain mode, which
still gets the C3 gain above.
Use the GGUF through your runtime's DFlash2 speculative-draft option beside a compatible Qwen3.8 27B target, for example in llama.cpp:
--spec-type draft-dflash --spec-draft-model Qwen3.8-27B-DFlash2-Marathon-C3-Q4_K_M.gguf \
--spec-draft-n-max 8 --spec-draft-ngl all --cache-type-k-draft q4_0 --cache-type-v-draft q4_0
Marathon downloads this file at an exact Hub revision and checks its size and SHA-256 before using it. Results can differ with another target, quantization, prompt mix, context length or sampler.
incoai/Qwen3.8-27B-DFlash2 @ dedf8df68adfb1afeaf7b7480c0a0243108177b4Busdriver/Qwen3.8-27B-Marathon-GGUF (IQ4_XS)The upstream drafter declares the Apache-2.0 license.