GigaChat3 10B-A1.8B DSpark Large Dev

Experimental development checkpoint. This model is not production-ready. On the tested RTX 5070 Ti setup, its DSpark path does not consistently accelerate the verifier. The checkpoint is published to make the experiment reproducible and to support further adapter and training work, not as a production speedup recommendation.

This repository contains a five-layer DSpark speculative draft model trained for the frozen ai-sage/GigaChat3-10B-A1.8B verifier. It is a speculative head, not a standalone language model.

This checkpoint was initialized from the companion Base Dev checkpoint. Two exact identity residual layers were appended to the trained three-layer draft, after which all five layers were fine-tuned together for one epoch on the same deterministic data snapshot.

The repository includes inference weights, complete optimizer and scheduler state, validation metrics, continuation metadata, exact launchers, TensorBoard events, and raw BS1/BS4 benchmark artifacts.

Model details

Usage with vLLM

Install a vLLM 0.23-compatible environment and the GigaChat3 vLLM plugin:

pip install "git+https://github.com/d0rj/gigachat3-vllm-plugin.git"

The plugin is discovered automatically through the vllm.general_plugins entry point; a shell-level VLLM_PLUGINS export is not required.

The explicit verifier-plus-draft form makes the number of proposed tokens visible and configurable:

vllm serve ai-sage/GigaChat3-10B-A1.8B \
  --host 127.0.0.1 \
  --port 8000 \
  --served-model-name gigachat3-dspark-large-dev-k3 \
  --trust-remote-code \
  --linear-backend triton \
  --max-model-len 2048 \
  --max-num-seqs 1 \
  --max-num-batched-tokens 4096 \
  --compilation-config '{"cudagraph_mode":"PIECEWISE"}' \
  --speculative-config '{"model":"d0rj/GigaChat3-10B-A1.8B.dspark-large-dev","method":"dflash","num_speculative_tokens":3}'

Set num_speculative_tokens to an integer from 1 through 7. K=3 and K=7 are benchmarked below. PIECEWISE CUDA Graph mode was used for the published measurements; full CUDA Graph capture is not compatible with the tested GigaChat3 MLA path. The Triton linear backend avoids the block-FP8 CUTLASS failure observed on the tested RTX 5070 Ti.

The vLLM method name remains dflash because DSpark uses the DFlash-family scheduler and proposer interface; the plugin dispatches to DSpark from DSparkDraftModel in the checkpoint configuration.

Request example:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gigachat3-dspark-large-dev-k3",
    "messages": [{"role": "user", "content": "Prove the fixed-point theorem."}],
    "max_tokens": 128,
    "temperature": 0
  }'

This checkpoint is not intended to be loaded as a standalone Transformers causal language model. The verifier remains responsible for exact output distribution and speculative verification.

Training

Training used online hidden-state extraction: the frozen verifier generated auxiliary states on demand and the complete hidden-state corpus was not persisted. The verifier and trainable draft shared one RTX 5070 Ti 16 GB GPU under WSL2. Layerwise verifier offload was used during training; it is not part of the benchmark serving configuration.

Data

The training snapshot contains exactly 200,000 deterministically selected rows from t-tech/T-Wix, pinned to revision 0059b0967915624069b5b4dc224ebe7ef7c041a6 and shuffled with seed 42. T-Wix is a Russian/English SFT mixture spanning general instructions, reasoning, mathematics, science, code, dialogue, and long-context tasks.

The prepared snapshot used a 90%/10% train/validation split. Conversations were rendered with the GigaChat3 chat template and limited to 8,192 tokens. The template contributes approximately 992 tokens even for a minimal exchange.

Exact preprocessing metadata is in training/data_config.json. T-Wix is ODC-BY-1.0 and its card notes that included third-party model outputs may have separate terms.

Hyperparameters

training/train.sh performs the three-to-five-layer expansion and starts fine-tuning. training/expand_dspark_layers.py preserves trained tensors and zero-initializes each added attention output and MLP down projection, making the new residual blocks exact identities at initialization. training/common_train.sh contains the complete online extraction and training command. train_command.txt records the resolved command and software versions from the run.

TensorBoard

The Base Dev and Large Dev TensorBoard event streams are included under tensorboard/base_dev and tensorboard/large_dev, allowing the initialization and fine-tuning stages to be inspected together:

tensorboard --logdir tensorboard --port 6006

Validation metrics

These are teacher-forced metrics from the released epoch-0 validation pass. Runtime speculative acceptance is lower because every later proposal depends on all preceding draft tokens being accepted.

Unrounded values are available in val_metrics.json.

vLLM performance

The benchmark compares the raw verifier with this checkpoint at K=3 and K=7. It uses the same chat-template-rendered prompt snapshots for all runs: 100 prompts each from MATH-500, GSM8K, HumanEval, MBPP, MT-Bench, and UltraChat. Each request uses greedy decoding, seed 0, a 2,048-token model limit, up to 128 output tokens, 4,096 maximum batched tokens, 0.85 GPU memory utilization, chunked prefill, the Triton linear backend, Inductor compilation, and PIECEWISE CUDA Graph mode. vLLM prefix caching remains enabled for every run. Eight warm-up requests are excluded per dataset.

BS1 uses concurrency/max-num-seqs 1; BS4 uses 4. Latency, TTFT, and ITL are reported as mean/p95 milliseconds. Acceptance rate is accepted draft tokens divided by all proposals; p1 is first-position acceptance; acceptance length is 1 + accepted_tokens / draft_steps.

Batch size 1

Batch size 4

No Large Dev configuration in this matrix exceeded the verifier baseline. The best observed speedup was 0.746x at BS1 and 0.583x at BS4. Large Dev generally raises teacher-forced and runtime acceptance slightly over Base Dev, but the two additional draft layers add enough compute to erase that gain in most tested workloads.

Training-held-out T-Wix subset

An additional short run uses 50 examples from indices [180000, 200000) of the prepared 200,000-row T-Wix Arrow dataset. The trainer used [0, 180000) for gradient updates, so these prompts were not training examples. They are, however, part of the validation split used for checkpoint selection and must not be interpreted as a fully untouched test set.

For each selected row, the benchmark removes the final supervised assistant answer and retains its chat-formatted prompt prefix. Only prefixes fitting the 1,920-token input budget without truncation are admitted. BS1 and BS4 use the same 50 source indices and prompt SHA-256 202dcb06e266cc65767bb0da94caf6adb3d919526f817336fda90c3381961483; all 250 measured requests in each campaign completed without errors. All other runtime settings match the main benchmark above.

The best T-Wix result is 0.707x at BS1/K=7. The five-layer checkpoint does not convert its higher K=7 acceptance length into a throughput advantage over Base Dev, and neither checkpoint accelerates the verifier on this subset.

Full reports, raw request samples, all recorded quantiles, prompt hashes, Prometheus snapshots, run configuration, and server logs are included under benchmarks.

Repository contents

• model.safetensors: selected five-layer DSpark weights.
• config.json, config.py: architecture and verifier linkage.
• optimizer_state_dict.pt: complete AdamW state.
• scheduler_state_dict.pt: scheduler state.
• training_state.json: epoch/global-step continuation metadata.
• train_command.txt: exact resolved training command and package versions.
• val_metrics.json: selected-checkpoint validation metrics.
• training/: dataset metadata and exact launchers.
• tensorboard/: Base initialization and Large fine-tuning TensorBoard event streams.
• benchmarks/: main BS1/BS4 campaigns plus the 50-prompt training-held-out T-Wix campaigns for the verifier, Base Dev, and Large Dev.

For optimizer-level continuation, restore the root checkpoint files as an epoch directory in a compatible Speculators checkpoint tree.

Limitations

• This is an experimental research artifact and is not production-ready.
• The tested DSpark configuration may be slower than the raw verifier. High training-time teacher-forced accuracy does not guarantee runtime acceleration.
• Performance depends strongly on hardware, workload, batch size, vLLM version, compiler cache state, and proposal count.
• The benchmark retains vLLM prefix caching. GigaChat3 prompts share a large chat-template prefix, so TTFT and aggregate throughput include prefix-cache benefits equally for baseline and DSpark runs.
• The published benchmark covers one RTX 5070 Ti 16 GB GPU under WSL2. Tensor parallelism, pipeline parallelism, multi-GPU serving, and other accelerators were not validated.
• The adapter targets the vLLM 0.23 development API used for these runs.
• GigaChat3 already exposes a native MTP speculative path. Native MTP was not included in this external-DSpark benchmark and should be evaluated for production use.
• K=7 matches the trained seven-proposal block; K=3 reduces draft work but changes the runtime block length.
• Generated outputs inherit the behavior, risks, and limitations of the verifier and T-Wix data.

License and attribution

The verifier is published under the MIT license. This checkpoint is distributed under MIT subject to the verifier's terms and all applicable dataset and third-party-data conditions. T-Wix is licensed under ODC-BY-1.0; consult its dataset card and source-specific terms before downstream use.

Training used the vLLM Speculators project with the GigaChat3 DSpark, online-extraction, confidence/Markov, and vLLM adapter extensions from the accompanying development branch.