DFlash2 drafter for DavidAU's Qwen3.8-Flash-Next Abliterated GSQ-RCO

A DFlash2 draft model fine-tuned for one target: DavidAU/Qwen3.8-Flash-Next-Abliterated-Uncensored-Turbo-Brilliance-50x-Reasoning-Instruct-GSQ-RCO-GGUF (IQ3_S). It starts from incoai/Qwen3.8-Flash-Next-DFlash2, which was trained on the official Qwen3.8-Flash-Next. It is not a standalone language model: it drafts tokens that the target verifies, so the output is the target's own (greedy output is identical with or without it).

v2 (2026-10-22): trained on 10x more of the target's own data, a wider draft vocabulary, and every llama.cpp quantization from F32 down to IQ4_XS, each measured. v1 (t3000-q8-dv.gguf) stays in place for existing configs.

Which file to use

The drafter is small next to the target (tens of GB; this is about 1 GB), so the smaller files save little memory, and on Apple Silicon they were not faster: the drafter's time is spent on its many small GPU steps, not on reading weights.

Files

What changed in v2

• Data. The target's own features and continuations, captured from the IQ3_S model in Strata: 2,907,544 rows from 3,094 requests (v1: 332,946 rows from 555), 1,024 tokens per answer at temperature 0.7, top-p 0.95, top-k 64, min-p 0.05. Every 25th request is held out. 1,872,374 training anchors.
• Training. From v1, 10,000 steps, batch 32, context 1,024 rows, AdamW lr 1e-5 (betas 0.9/0.95), cosine decay with 100 warm-up steps. Picked by held-out score among checkpoints saved every 1,000 steps: step 9,000. Learning rates 2e-5 and 5e-5 were worse in 600-step probes; 1e-4 diverged; context 2,048 in training diverged.
• Draft vocabulary. v1 drafted from 40,525 token ids; 5.3% of the tokens this target writes were outside that list, so they could never be drafted. v2 adds the 435 ids the target writes most among those (Strata's native drafter takes at most 40,960). Held-out coverage 94.2% to 95.2%. Nothing was removed, so Chinese and other scripts the list already held are unaffected.
• Quantization. An importance matrix from the drafter's own inputs on 6,400 training anchors (all 50 matrices, including the output head and both selector codebooks), then llama.cpp's llama-quantize per type.

Measured

Quality of every file

8,192 held-out anchors from 124 held-out requests, context 2,048 rows (what the server uses), drafting only from the draft vocabulary (as on the Mac). Each GGUF is read back through ggml's own dequantization, so these are the weights llama.cpp computes with.

• Accepted / 7: mean drafts of a 7-token block that match the target. First 3: the same over the first 3, which is what --spec 4 uses.
• KL: KL(BF16 drafter || this file) of the draft-vocabulary distributions, mean over the 7 positions (first 2,048 anchors).
• Top-1 agree: how often this file's top token equals the BF16 drafter's. Same draft: how often its whole 3-token draft (the selector's walk) equals the BF16 drafter's.

Differences under about 0.01 in "Accepted / 7" are within the noise of 8,192 anchors from 124 requests; the KL and agreement columns separate the files more finely. The v1 row is v1's weights scored with the v2 draft vocabulary. On the 1,024-anchor subset used during training (context 1,024): incoai drafter 2.824 / 1.793, v1 2.948 / 1.849, v2 2.976 / 1.873.

Speed on a Mac

Lossless: with each Mac file (Q8_0, mac-Q5_0, mac-Q4_0, mac-IQ4_NL, mac-IQ4_XS) the target's greedy output was byte-identical to its output with MTP on 6 of 6 prompts (evidence/v2/greedy/). The drafter changes speed, never text.

Apple M4 Max, 64 GB, macOS 26.5, expert arena 42 GB, greedy, --spec 4 --spec-min-p 0.75 (MTP: --spec 3, min-p 0.7), one run each, 2026-10-22. Short: decode tok/s over 3 prompts. Long: a ~25K-token code prompt, then a follow-up on the cached prefix.

Read this honestly: every drafter row is within about 1.5 tok/s of the others, which is run-to-run noise here (macOS swap grew from 2.5 to 5.3 GB during the run). v2's higher acceptance shows offline (3.025 vs 2.970) but is too small to separate from v1 in one serving run at --spec 4, which drafts only 3 tokens a window. The 4-bit files are not faster. Against Strata's MTP default on this target, both DFlash2 versions were 0-2 tok/s faster in this run (v1's own runs, below, measured 1.3-3.7). Not measured: sampling at temperature > 0, other Macs, CUDA, HIP.

Window size, repeated (recommended settings)

The same workloads, Q8_0 only, 3 interleaved rounds (v1, v2, v2, v2 per round), mean ± standard deviation:

Use v2 with --spec 5. It is the only setting at least as fast as v1 on all three workloads: about +0.5 tok/s on short prompts and +0.8 on the long one, and the same on the cached follow-up. At --spec 4, v2 was 2 tok/s slower than v1 on the follow-up in all three rounds. v2 is better at the later draft positions, so it gains from drafting 4 tokens a window instead of 3. Raw rows: evidence/v2/mac-speed-repeated.tsv.

Native Metal drafter against llama.cpp

dflash_native_parity_test, real target features, 2,200 context rows, 300 drafts of 1-7 tokens: Q8_0, Q5_0, Q4_0, IQ4_NL and IQ4_XS each drew the same 300 of 300 draft chains as llama.cpp's DFLASH graph. Confidences agreed within 0.0023, except one Q5_0 confidence at 0.045 (--spec-min-p reads it).

Use with Strata (Apple Silicon)

Needs the Metal engine built with -DSTRATA_DFLASH=ON (opt-in; Strata PR #3457). In the config's args, in place of --mtp …:

--spec 5 --dflash DFlash2-v2-Q8_0.gguf --spec-min-p 0.75

The mac/ files and the non-Q8_0 gguf/ files need multi-format support in the native drafter, which PR #3457 does not have yet: the PR's drafter reads Q8_0 only and refuses the others with a clear error. The K-quant files (Q6_K, Q5_K_*, Q4_K_*) load in llama.cpp's DFLASH graph, not in Strata's native drafter.

With llama.cpp

Any gguf/ file loads as a DFLASH draft model in a llama.cpp build with DFlash2 support; all 15 were loaded and drafted with llama.cpp's DFLASH graph (dflash_drafter_test, no NaN).

v1 notes

The MTP baseline is not from the target model

DavidAU's GGUF has no MTP (multi-token prediction) tensors: 0 of its 1224 tensors. The MTP layer in the comparisons comes from a different model: the mtp.* block (31 tensors) of the official Qwen/Qwen3.8-Flash-Next BF16 checkpoint, revision de4b8e4d, extracted by Strata's setup and packed by Strata's tools/mtp_pack.py as mtp-q2_0.gguf (experts Q2_0, the rest BF16/F32). This is what Strata runs for this GGUF by default. It was trained with the official model, not this abliterated fine-tune, and its experts run at Q2_0, so the comparison favors this drafter. See evidence/mtp-provenance/.

v1 results

v1: 3,000 steps on 332,946 rows. Apple M4 Max, greedy, against that MTP layer (spec 3, min-p 0.7): short prompts 49.7 vs 48.1 tok/s, 29K prompt 51.7 vs 50.4, 102K prompt 50.5 vs 46.8 (cold). Details in evidence/.

Special thanks

Special thanks to incoai for DFlash2 and the Qwen3.8-Flash-Next-DFlash2 drafter this one is fine-tuned from: the architecture, the candidate selector and every starting weight are theirs (blog, code). This work only adapts their drafter to one target model. Thanks also to DavidAU for the target model.

Special thanks to ACOTech for providing the machine behind this drafter. Every step ran on it, locally: capturing the target's features, both fine-tunes, the held-out evaluations, the quantizations and every benchmark on this page. The machine is a Mac Studio with an Apple M4 Max, 64 GB of unified memory. Without it this drafter would not exist.

How this was made

The data capture, fine-tune, quantization, evaluation, benchmarks and this page were done with Claude Code running Claude Opus 5.5, directed and reviewed by hotocooo.

License

Apache-2.0, like the incoai drafter and the target model.