Thanks to the SpecForge framework for their foundational contributions.
Model Overview
Ling-3.1-Flash-DSpark is a self-trained DSpark draft model designed to accelerate Ling-3.1-Flash inference through speculative decoding.
Performance & Acceleration
We evaluate 5-token and 7-token drafting configurations across mathematical reasoning, code generation, dialogue, and knowledge benchmarks.
Both drafting configurations outperform the AR and MTP baselines across all evaluated benchmark–concurrency combinations.
Evaluation Configuration:
- Target model: Ling-3.1-Flash
- Decoding temperature:
T = 0(greedy decoding) - Draft tokens: 5 or 7
- Concurrency: 1, 8, 16, and 32
- Benchmarks: GSM8K, MATH500, MBPP, HumanEval, MT-Bench, and C-Eval
- Metrics: Mean accepted length (
τ) and throughput (tokens/s)
Results:
5 Draft Tokens
7 Draft Tokens
Quick Start
Requirements
- NVIDIA GPU
- CUDA 12.0+
- PyTorch 2.0+
Installation
pip install sglang==0.5.22
Inference with SGLang
python3 -m sglang.launch_server \
--model-path /models/Ling-3.1-flash \
--host 0.0.0.0 \
--port 30010 \
--mem-fraction-static 0.82 \
--trust-remote-code \
--tp 8 \
--ep 8 \
--speculative-algorithm DSPARK \
--speculative-draft-model-path AQ-MedAI/Ling-3.1-flash-DSpark \
--speculative-dspark-block-size 5
For the 7-token configuration, change:
--speculative-dspark-block-size 7
Citation
If you use this model in your research or application, please cite the following:
@misc{ling31-flash-dspark,
title={Ling-3.1-Flash-DSpark: Accelerating Inference with DSpark},
author={Ant AQ Team},
year={2026},
}