AEON DFlash — Qwen3.6 / Ornith 35B-A3B

!AEON Qwen — Supreme Being of the Digital Cosmos

A DFlash speculative-decoding drafter for Qwen/Qwen3.6-35B-A3B and its derivatives — including Ornith-1.0-35B-AEON-Ultimate-Uncensored (a light RL post-train of that base). ~0.9 GB. Gives ~1.9× single-stream decode on a DGX Spark with no quality loss (lossless speculative decoding: the target model verifies every token).

What makes it different

This drafter is 8 layers of full attention — deliberately no sliding-window attention (SWA). That single design choice is the difference:

| | This drafter (AEON) | Typical SWA drafter (e.g. z-lab) |

|---|---|---|

| Layers | 8 × full attention | 6 × (5 sliding + 1 full), window 4096 |

| KV page size | aligns with the target's pages | larger (windowed) → mismatch |

| Serves on the AEON vLLM image | yes, no patch | needs a kv_cache_utils patch |

Why the no-patch part matters. Qwen3.6-35B-A3B / Ornith is a hybrid stack — full attention + GatedDeltaNet (Mamba) recurrent state. vLLM unifies KV-cache page sizes across layer types by scaling block_size, but Mamba/GDN state pages are shape-based and don't scale. A drafter with SWA layers introduces a larger KV page that can't be reconciled with the GDN pages → vLLM aborts at engine init with unify_kv_cache_spec_page_size … AssertionError (see issue #1). An all-full-attention drafter has no oversized page, so it loads cleanly on the stock AEON image — a true drop-in.

It also targets the right hidden states for this stack: DFlash reads target layers [1, 10, 19, 28, 37] of the 40-layer 35B-A3B model.

A note on KV cache (and why SWA doesn't help here)

This model serves with BF16 KV cache, and a sliding-window drafter does not change that. The BF16-KV requirement comes from two places, neither of which is the drafter's window type:

1. the vision tower (non-causal vision attention → FA2 backend only → no FP8 KV), and

2. DFlash's non-causal verification (block-parallel → FA2/FLEX only → no FP8 KV).

KV dtype is a property of the main model's attention backend, not the drafter's layers. SWA changes the drafter's own (tiny, windowed) KV footprint only — it can't unlock FP8 KV for the target.

Usage

Point any Qwen3.6-35B-A3B / Ornith deploy's --speculative-config at this drafter:

  --speculative-config '{"method":"dflash","model":"/drafter","num_speculative_tokens":6}'

Validated on a DGX Spark (GB10) with the Ornith NVFP4 QuickStart. ⚠️ Keep --max-num-seqs ≤ 16 with DFlash on the Spark (unified-memory limit — see the QuickStart).

Performance

Head-to-head on Ornith-1.0-35B NVFP4 (DGX Spark GB10 · n=6 · max-num-seqs 16 · single stream):

|---|---|---|---|---|

Speed is effectively tied (c=1 within ~2% — noise; this drafter is slightly ahead at c=8 aggregate). This drafter wins on acceptance — its 8 full-attention layers draft more accurately (3.71 vs 3.35 tokens accepted per step; 45% vs 39%) than the SWA variant's 6 layers, which trade accuracy for a cheaper forward pass. Combined with no patch required and no padded-Mamba-page overhead, it's the recommended default. Both are lossless (the target verifies every token); ~1.9× over plain decode either way.

Credits

Drafter: AEON-7. Method: DFlash speculative decoding.
KV page-size diagnosis + the patch for SWA-drafter compatibility: @newjordan (issue #1 / PR #2).