Kimi K2.5 DFlash (GGUF)

This repository contains a GGUF format conversion of the Kimi K2.5 DFlash speculative decoding draft model, including quantized versions.

Model Details

Files

What is DFlash?

DFlash (DeepFlash) is a speculative decoding technique that uses a lightweight draft model to predict tokens, which are then verified by the full target model (Kimi K2.6 / K2.5). This speeds up inference by 2-3x on average.

Usage

With oxidize (Rust inference engine)

The GGUF files are compatible with the oxidize inference engine (PR #1+), which has native DFlash speculative decoding support.

oxidize --model /path/to/Kimi-K2.5-DFlash-Q4_0.gguf --speculative-config ...

With llama.cpp / other GGUF tools

This model uses the custom dflash-draft architecture. Standard llama.cpp does not natively support DFlashDraftModel inference. Use oxidize or wait for upstream support.

Conversion & Quantization Details

• Source: z-lab/Kimi-K2.5-DFlash (SafeTensors, bfloat16)
• Converter: Custom Python script using gguf library + safetensors.torch
• Quantizer: Python script using gguf.quants.quantize() with GGMLQuantizationType
• Weight Mapping: HuggingFace names (layers.N.self_attn.q_proj.weight) → oxidize GGUF names (blk.N.attn_q.weight)
• Metadata: Includes general.architecture=dflash-draft and all model hyperparameters
• 1D tensors (norm weights, biases) are kept at F32 for accuracy

Limitations

• No lm_head or embed_tokens weights: DFlash draft models reuse the target model's token embeddings
• This is a draft model only — it requires a full target model (e.g. moonshotai/Kimi-K2.6) for speculative decoding

License

Apache 2.0 — same as the original z-lab/Kimi-K2.5-DFlash model.