Ling-3.0-Tiny-DSpark-50K

Version: v0 (base checkpoint)

A DSpark speculative-decoding draft for inclusionAI/Ling-3.0-tiny, trained with SpecForge and served with SGLang. This is the base checkpoint: it carries its own rank-256 vanilla Markov head and contains no donor-derived weights. The separately published Ling-3.0-Tiny-DSpark is a variant that blends 25% of a larger sibling model's Markov output bias into this head before 3,000 further training steps.

Not an official inclusionAI release, and not a standalone language model. It is a draft that must be loaded next to the Tiny target.

Created by Jay Yun — GitHub · LinkedIn.

Checkpoint

The draft shares the target's token embedding and lm_head, which is why its hidden width equals the target's. The fc projector concatenates the five selected target-layer hidden states (5 × 1536) into the draft's 1536 width. Training was a cumulative 50K-step horizon in BF16 (a 30K warm start plus 20,000 further steps); the repository name records that horizon.

Acceptance length and throughput — limited pilot

One H200, static verify-all, temperature 0, reasoning disabled, seed 42, 20 exact-chat prompts, maximum 256 generated tokens, concurrency 1. Target repeatability and exact-text equality against the target were checked.

Acceptance length is 1 + correct / verification rounds, where the 1 is the target bonus token. Observed speedup over the target-only run was about 2.49x in this single pilot, not a repeated or production-wide claim. Averages above are from one 20-prompt run with single timings.

Exact-count histogram for 0 through 7 accepted proposals: [607, 431, 228, 130, 107, 59, 47, 80]. Histogram bins are not position acceptance probabilities.

Serving

The recorded run used lmsysorg/sglang:dev-cu12-Ling-3.0-tiny, FA3, deterministic inference, CUDA graphs and radix cache disabled, context length 4096, SGLANG_RAGGED_VERIFY_MODE=static, DSPARK block size 7 and 8 draft-token slots. The image tag is mutable, so exact runtime packaging is a release checklist item.

hf download inclusionAI/Ling-3.0-tiny \
  --revision a2ee06c0f2de5b171701aee7f73f70a1da75483b --local-dir ./ling-tiny
hf download jayyun98/Ling-3.0-Tiny-DSpark-50K --local-dir ./ling-tiny-dspark-50k

docker run --rm --gpus all --ipc=host \
  -p 127.0.0.1:30000:30000 -v "$PWD:/models:ro" \
  -e SGLANG_RAGGED_VERIFY_MODE=static \
  lmsysorg/sglang:dev-cu12-Ling-3.0-tiny \
  python3 -m sglang.launch_server \
  --model-path /models/ling-tiny --trust-remote-code \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path /models/ling-tiny-dspark-50k \
  --speculative-dspark-block-size 7 --speculative-num-draft-tokens 8 \
  --served-model-name ling-tiny --reasoning-parser ling3 \
  --attention-backend fa3 --context-length 4096 \
  --max-running-requests 1 --max-total-tokens 8192 \
  --chunked-prefill-size -1 --disable-radix-cache --disable-cuda-graph \
  --enable-deterministic-inference --host 0.0.0.0 --port 30000

Use the Tiny target's tokenizer and chat template with thinking explicitly disabled. Do not load this draft alone as a Transformers text-generation model. SGLang selects the draft class from the architectures field; the required markov_rank / markov_head_type / mask_token_id values are promoted to top-level config keys, and SGLang reads them there.

Limitations

• Only 20 prompts at 256 tokens and concurrency 1; no independent fresh-64 long generation gate, no concurrency 4/8 sweep, and no repeated timings.
• No standardized GSM8K / MATH / HumanEval result is claimed.
• 20/20 exact-text parity is evidence for this pilot only, not a universal correctness guarantee, and the recorded run was not a production soak.
• An earlier H100 gate recorded a different acceptance value under non-deterministic inference settings and a different target revision. It is not comparable to the table above and is not used here.
• This checkpoint is the parent of the blended sibling. It is published for attribution and reproducibility of that comparison, not because it is the stronger of the two.

License and attribution

These weights are trained by us and contain no weights from a third-party draft; the license follows the MIT-declared base target. Training and serving toolchains (SpecForge, SGLang, DSpark) carry their own Apache-2.0 attributions, which are separate from the weight license. Consult the upstream terms for your use.

Upstream sources: Tiny target and SpecForge.