A DFlash speculative-decoding drafter for Qwen/Qwen3.6-35B-A3B and its derivatives — including Ornith-1.0-35B-AEON-Ultimate-Uncensored (a light RL post-train of that base). ~0.9 GB. Gives ~1.9× single-stream decode on a DGX Spark with no quality loss (lossless speculative decoding: the target model verifies every token).
This drafter is 8 layers of full attention — deliberately no sliding-window attention (SWA). That single design choice is the difference:
| This drafter (AEON) | Typical SWA drafter (e.g. z-lab) | |
|---|---|---|
| Layers | 8 × full attention | 6 × (5 sliding + 1 full), window 4096 |
| KV page size | aligns with the target's pages | larger (windowed) → mismatch |
| Serves on the AEON vLLM image | yes, no patch | needs a kv_cache_utils patch |
Why the no-patch part matters. Qwen3.6-35B-A3B / Ornith is a hybrid stack — full attention + GatedDeltaNet (Mamba) recurrent state. vLLM unifies KV-cache page sizes across layer types by scaling block_size, but Mamba/GDN state pages are shape-based and don't scale. A drafter with SWA layers introduces a larger KV page that can't be reconciled with the GDN pages → vLLM aborts at engine init with unify_kv_cache_spec_page_size … AssertionError (see issue #1). An all-full-attention drafter has no oversized page, so it loads cleanly on the stock AEON image — a true drop-in.
It also targets the right hidden states for this stack: DFlash reads target layers [1, 10, 19, 28, 37] of the 40-layer 35B-A3B model.
This model serves with BF16 KV cache, and a sliding-window drafter does not change that. The BF16-KV requirement comes from two places, neither of which is the drafter's window type:
KV dtype is a property of the main model's attention backend, not the drafter's layers. SWA changes the drafter's own (tiny, windowed) KV footprint only — it can't unlock FP8 KV for the target.
Point any Qwen3.6-35B-A3B / Ornith deploy's --speculative-config at this drafter:
--speculative-config '{"method":"dflash","model":"/drafter","num_speculative_tokens":6}'
Validated on a DGX Spark (GB10) with the Ornith NVFP4 QuickStart. ⚠️ Keep --max-num-seqs ≤ 16 with DFlash on the Spark (unified-memory limit — see the QuickStart).
Head-to-head on Ornith-1.0-35B NVFP4 (DGX Spark GB10 · n=6 · max-num-seqs 16 · single stream):
| Drafter | c=1 decode | Accept length | Draft accept rate | Patch needed |
|---|---|---|---|---|
| AEON (8L full-attn) — this repo | 72.9 tok/s | 3.71 | 45.1% | none |
| z-lab (6L SWA) | 74.4 tok/s | 3.35 | 39.2% | KV page-size hotfix |
Speed is effectively tied (c=1 within ~2% — noise; this drafter is slightly ahead at c=8 aggregate). This drafter wins on acceptance — its 8 full-attention layers draft more accurately (3.71 vs 3.35 tokens accepted per step; 45% vs 39%) than the SWA variant's 6 layers, which trade accuracy for a cheaper forward pass. Combined with no patch required and no padded-Mamba-page overhead, it's the recommended default. Both are lossless (the target verifies every token); ~1.9× over plain decode either way.
AEON-7 models, drafters and tools are built and trained independently, on my own hardware. If they're useful to you, please consider supporting development.
Patreon wrongfully shut down our account without notice and never responded to our appeals, so we built something even better. You can now support the work on LotusMake: buy us a Lotus Tea, fund GPU hours, back a project you want to see done, or subscribe for early and full access to publications while supporting what we build. You can become a creator there too: join, apply to become a creator, and get paid for the work you do in public.
AEON FORGE on LotusMake → lotusmake.com/c/aeon-forge · Subscribe for early and full access
Milestones unlock bigger work: reaching 500 paid supporters will fund fine-tuning larger models and bigger project releases. Subscribers also get early access to new releases, such as the AEON DFlash2 drafter.
License: MIT.