Whittle 35B-A3B DFlash2

A focused draft for Whittle.

526M parameters  ·  F16 GGUF  ·  v1.1.0 Nightly  ·  Apache-2.0

Download GGUF · Training data · Release notes

---

Nightly / interim checkpoint — November 12, 2026. Training is still running. v1.1.0 packages the best saved checkpoint at the release snapshot: continuation step 4,978, selected by validation loss. It is an experimental progress release, not the final training result.

A DFlash2 drafting adapter for speculative decoding with Whittle-Qwen-3.8-35B-A3B, adapted to its Q5_K_M GGUF. Run it alongside the separate Whittle target in a compatible DFlash2 runtime. The target stays frozen while all draft parameters are fine-tuned.

At a glance

What's improved

The same 256-example validation monitor is used throughout this continuation. The baseline is the v1.0.0 parent evaluated on this monitor; these figures are offline draft-training metrics.

At the snapshot, 78 of 176 first-pass shards had completed, covering 19,912 usable training traces. Checkpoint selection follows the lowest monitor loss; accuracy can fluctuate between checkpoints. Full numeric history: EVALUATION.json and training/validation-history.jsonl.

A new tokens/s or runtime-acceptance benchmark has not been run for v1.1.0. Better validation metrics are promising but do not establish a serving-speed gain or better final answers. The reserved test benchmark remains pending.

Training data and recipe

This continuation additionally uses DFlash DeepSeek TraceMix: 50,000 RU/EN task contexts, split into 45,000 training, 2,500 validation and 2,500 test tasks. It mixes programming and project-code work, text tasks, reasoning and agent/tool-use contexts. Earlier adaptation used DFlash TraceMix Lite and prompts selected from Tulu 3 SFT Mixture.

Whittle Q5 supplies the training completions and hidden features. DeepSeek Flash acts as a training mentor, reviewing aggregate metrics and choosing whether to continue or adjust the learning rate. DeepSeek-generated agent histories are part of the broader dataset; the 50,000 contexts are not 50,000 DeepSeek answers.

All 526M draft parameters are updated with AdamW, batch 4, a 384-token feature context, base LR 5e-6 and projection LR 1.5e-5, using FP32 master weights and FP16 autocast. Each shard is validated before commit, with automatic retry of recoverable FP16 gradient overflows. See TRAINING.md, DATA_SOURCES.md and PROVENANCE.json.

Files and use

• Whittle-35B-A3B-DFlash2-v1.1.0-F16.gguf — inference draft, with author, version, license and nightly status embedded in GGUF metadata.
• tensors/best-model.safetensors — the same 96 trained tensors, in PyTorch module naming, plus tensor inventory. No optimizer state or target weights.
• training/v1.1.0/ — portable shard trainer, AMP retry helper and recipe; training/validation-history.jsonl records the monitor trajectory.

Use the compatible Linux/NVIDIA DFlash2 runtime and a separate Whittle Q5 target. USAGE.md lists the launch parameters and download verification. Checksums are in SHA256SUMS.

Previous release measurements and limitations

The v1.0.0 local long-generation test recorded 74.08 decode tokens/s, 67.62 end-to-end tokens/s and 59.53% draft acceptance; its no-draft control recorded 53.60 decode tokens/s. Hardware: V100 32 GB for target computation and draft, RTX 3060 12 GB for target n-gram memory; custom llama.cpp runtime, at most three proposed tokens and BF16 draft KV. These are historical results and do not measure v1.1.0. Archived evaluation.

The older test did not confirm exact greedy equivalence, failed both 64K marker checks with the draft, and did not test a filled 256K context. Validation pools were reused during selection. Those open issues have not been re-evaluated for this nightly. Other target quantizations, stock llama.cpp and Apple Metal remain unvalidated. Anonymization of publication text does not imply unlearning of earlier training data; see PRIVACY.md.

main contains v1.1.0 Nightly. The existing v1.0.0 branch preserves the previous release.

---

_{Adaptation by Egor Prokhorov · Published by biscet · License & notices}