Ripple-65B-Preview — DFlash2 Draft, SAGE REXL3
EXL3-quantized DFlash2 speculative-decoding drafts for
Ripple-65B-Preview
(target pack:
vcruz305/Ripple-65B-Preview-SAGE-REXL3-5.41bpw).
Quantized from Ricursive's BF16 draft
(ricursiveAI/Ripple-65B-Preview-DFlash2)
by vcruz305 with SAGE, our own
mixed-precision quantization method for EXL3. Tensors use different EXL3 K
values; the kernels read K from each trellis tensor at load.
Three bitwidths: 4.55bpw (recommended), 3.60bpw, 3.08bpw —
one subdirectory per pack. 5 layers, selector_rank 64, top_k 32, target
layers [1, 20, 39, 58, 77].
Serving: production recipe on SGLang 0.5.19 with DFlash2 speculative decoding:
- Recipe repository:
vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe - SGLang EXL3 plugin:
vcruz305/sglang-exl3
Measured serving speed (SGLang 0.5.19 + sglang-exl3)
One NVIDIA RTX PRO 6000 Blackwell 96 GB (sm_120), TP=1, 5.41bpw target +
these drafts, --context-length 131072, --mem-fraction-static 0.82, CUDA graphs
enabled, streaming TTFT with stream_options={"include_usage": True}. Prompt sizes
are actual tokenized counts (403 / 1564 / 3152).
| Draft | VRAM | Greedy decode 403 / 1564 / 3152 tok | t=0.6 decode 403 / 1564 / 3152 tok |
Accept len |
|---|---|---|---|---|
4.55bpw EXL3 (recommended) |
2.19 GB | 64.8 / 44.9 / 49.3 tok/s | 53.1 / 41.2 / 51.3 tok/s | 2.05 – 3.55 |
3.60bpw EXL3 |
1.88 GB | 28.7 / 25.2 / 23.1 tok/s | 28.5 / 25.0 / 24.9 tok/s | 1.50 – 3.10 |
3.08bpw EXL3 |
1.66 GB | 41.6 / 24.6 / 19.0 tok/s | 24.6 / 31.8 / 22.8 tok/s | 1.90 – 2.90 |
| BF16 (upstream reference) | 7.60 GB | 59.3 – 60.9 / 41.5 / 45.5 tok/s | 31.8 / 33.2 / 42.9 tok/s | 1.30 – 3.20 |
Cold prefill with the 4.55bpw draft: 1,096.5 – 1,118.6 / 1,672.6 – 1,674.7 / 1,796.3 – 1,802.2 tok/s;
KV pool: 117,376 full-attention / 93,888 sliding-window tokens (vs 97,408 full-attention tokens with the BF16 draft, saving 5.41 GB VRAM). Full tables, launch flags, and tuning knobs live in vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe.
Pack
| Item | Value |
|---|---|
| Format | EXL3 (trellis) — one subdirectory per draft |
| Drafts | 4.55bpw/, 3.60bpw/, 3.08bpw/ |
| Pack size on disk | 2.7 / 1.9 / 1.7 GB |
quant_method |
exl3 |
| Draft architecture | 5 layers, selector_rank 64, top_k 32, target layers [1, 20, 39, 58, 77] |
Serve (SGLang 0.5.19 + sglang-exl3)
Clone vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe and run:
bash scripts/setup.sh
bash scripts/download.sh
bash scripts/serve.sh
Direct launcher invocation:
python3 scripts/serve_ripple.py \
--model-path /path/to/Ripple-65B-Preview-SAGE-REXL3-5.41bpw \
--speculative-draft-model-path /path/to/Ripple-65B-Preview-DFlash2-SAGE-REXL3/4.55bpw \
--context-length 131072
For an unquantized BF16 draft instead, pass --speculative-draft-model-quantization unquant
and point --speculative-draft-model-path at the upstream BF16 draft directory.
About SAGE
SAGE is our own mixed-precision quantization method for EXL3. The method is not published. If a tensor is EXL3, its K is self-describing on the trellis.
Source model (Ricursive, not this pack)
The draft is derived from Ripple-65B-Preview-DFlash2 by Ricursive Intelligence; the target model is Ripple-65B-Preview. Architecture and benchmark rows on the upstream cards are Ricursive's.
License
Apache 2.0, same as Ripple-65B-Preview-DFlash2. Cite Ricursive for the base model/draft. This repository is the EXL3 draft pack only.