Ripple-65B-Preview — SAGE REXL3 5.41 bpw
Loadable ExLlamaV3 / SGLang EXL3 pack of ricursiveAI/Ripple-65B-Preview.
Quantized by vcruz305 with SAGE, our own
mixed-precision quantization method for EXL3. Tensors use different EXL3 K
values; the kernels read K from each trellis tensor at load.
Sized for a 48 GB-class device without the draft (~42.1 GiB GPU-resident weights, 45.2 GB pack, text-only). This 5.41 bpw quantization preserves 99.34% Top-1 agreement (0.0005 mean KLD) with Ricursive's official BF16 release (ricursiveAI/Ripple-65B-Preview).
Serving: production recipe on SGLang 0.5.19 with DFlash2 speculative decoding (4.55bpw EXL3 draft):
- Recipe repository:
vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe - SGLang EXL3 plugin:
vcruz305/sglang-exl3 - Quantized DFlash2 drafts:
vcruz305/Ripple-65B-Preview-DFlash2-SAGE-REXL3(4.55bpw,3.60bpw,3.08bpw)
Measured serving speed (SGLang 0.5.19 + sglang-exl3)
One NVIDIA RTX PRO 6000 Blackwell 96 GB (sm_120), TP=1, this pack +
4.55bpw EXL3 DFlash2 draft, --context-length 131072, --mem-fraction-static 0.82,
CUDA graphs enabled, streaming TTFT with stream_options={"include_usage": True}.
Prompt sizes are actual tokenized counts (403 / 1564 / 3152).
| Metric | 403 tok | 1564 tok | 3152 tok |
|---|---|---|---|
| Cold prefill tok/s | 1,096.5 – 1,118.6 | 1,672.6 – 1,674.7 | 1,796.3 – 1,802.2 |
Pure decode greedy (t=0.0) tok/s |
64.8 | 44.9 | 49.3 |
Pure decode sampled (t=0.6) tok/s |
53.1 | 41.2 | 51.3 |
| DFlash2 accept length (tok/round) | 2.05 – 3.55 | 1.40 – 2.70 | 2.30 – 2.70 |
KV pool: 117,376 full-attention / 93,888 sliding-window tokens (30.11 GB hybrid SWAKVPool); draft weights 2.19 GB; ~83 GB total VRAM used. Full launch scripts, benchmarks, and tuning knobs live in vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe.
SixCat evaluation
SixCat (challenge-v1, thinking=on, temp=0.6, seed=1, 20 items/category, DFlash2 speculative decoding enabled):
- Overall (
5.41 bpw): 88.89% (118/120scored) — Knowledge83.3%, Math100.0%, Truthfulness80.0%, Instruction Following85.0%, Coding95.0%, Tool Use90.0%.
Measured quality
Teacher-forced token-level agreement with Ricursive's official BF16 release (ricursiveAI/Ripple-65B-Preview), full-vocabulary KL divergence per token, on held-out sequences (assistant-turn tokens, 55,535 scored):
| Pack | Resident footprint | KL vs official BF16 release (mean) | KL vs official BF16 release (p50) |
Top-1 Agreement | Top-5 Agreement | Top-5 Overlap |
|---|---|---|---|---|---|---|
| this pack, 5.41 bpw | ~42.1 GiB | 0.0005 | 0.000004 | 99.34% | 99.998% | 95.63% |
| Official BF16 release | native | 0 | 0 | 100% | 100% | 100% |
Per-category breakdown
| Category | KL vs official BF16 release (mean) | KL vs official BF16 release (p50) |
Top-1 Agreement | Answer-Only Top-1 | Top-5 Overlap |
|---|---|---|---|---|---|
| general | 0.0003 | 0.000001 |
99.54% | – | 95.32% |
| prose_narrative | 0.0005 | 0.000020 |
99.39% | – | 96.68% |
| multilingual | 0.0011 | 0.000041 |
98.97% | – | 96.12% |
| reasoning | 0.0003 | 0.000001 |
99.57% | – | 95.58% |
| code | 0.0002 | 0.000002 |
99.70% | – | 96.09% |
| ifeval | 0.0001 | 0.000001 |
99.58% | – | 93.58% |
| gpqa_stem | 0.0013 | 0.000324 |
98.12% | – | 97.37% |
| swe_patch | 0.0002 | 0.000002 |
99.74% | – | 96.01% |
| math | 0.0003 | 0.000003 |
99.33% | – | 94.72% |
| cyber | 0.0004 | 0.000001 |
99.58% | – | 94.58% |
| mmlu_pro | 0.0002 | 0.000001 |
99.31% | – | 94.71% |
Held-out & long-context suites (assistant-turn tokens)
| Suite | Context | Assistant-turn tokens | KL vs official BF16 release (mean) | KL vs official BF16 release (p50) |
Top-1 Agreement | Top-5 Agreement | Answer-Only Top-1 | Top-5 Overlap |
|---|---|---|---|---|---|---|---|---|
| wiki1100 | 1,100-token prefix | 890 | 0.001269 | 0.000467 |
98.54% | 100% | – | 97.64% |
| long17k | 17,408-token long context | 10,654 | 0.001272 | 0.000438 |
98.70% | 100% | – | 97.19% |
Pack
| Item | Value |
|---|---|
| Format | EXL3 (trellis) |
| Profile | 5.41 bpw SAGE mixed-K |
| Resident footprint | ~42.1 GiB (text-only) |
| Shards | 10 (model-00001-of-00010.safetensors .. model-00010-of-00010.safetensors) |
| Index | model.safetensors.index.json |
quant_method |
exl3 |
model_type |
ricursive |
Serve (SGLang 0.5.19 + sglang-exl3)
Clone the recipe repository and run the setup, download, and serve scripts:
git clone https://github.com/vcruz305/Ripple-65B-Preview-EXL3-SGLang-recipe.git
cd Ripple-65B-Preview-EXL3-SGLang-recipe
# 1. Build ExLlamaV3 from source and install SGLang 0.5.19 + vcruz305/sglang-exl3
bash scripts/setup.sh
# 2. Download this 5.41bpw target pack + the 4.55bpw EXL3 DFlash2 draft pack
bash scripts/download.sh
# 3. Launch the OpenAI-compatible server on http://127.0.0.1:30000/v1 (DFlash2 ON, 131K context)
bash scripts/serve.sh
Direct launcher invocation:
python3 scripts/serve_ripple.py \
--model-path /path/to/Ripple-65B-Preview-SAGE-REXL3-5.41bpw \
--speculative-draft-model-path /path/to/Ripple-65B-Preview-DFlash2-SAGE-REXL3/4.55bpw \
--context-length 131072
Load (ExLlamaV3)
from exllamav3 import Config, Model, Tokenizer
cfg = Config.from_directory("vcruz305/Ripple-65B-Preview-SAGE-REXL3-5.41bpw")
model = Model.from_config(cfg)
model.load()
tokenizer = Tokenizer.from_config(cfg)
About SAGE
SAGE is our own mixed-precision quantization method for EXL3. The method is not published. If a tensor is EXL3, its K is self-describing on the trellis.
Source model (Ricursive, not this pack)
Ripple-65B-Preview is Ricursive Intelligence's dense 65B autoregressive model: 80 layers, hidden size 5120, GQA with 80 query / 8 KV heads, gated attention, value-residual connections, and a hybrid sliding-window / full-attention pattern (sliding window 1024; full attention every 5th layer) with 2,097,152 max positions. Architecture, sampling defaults, and any benchmark rows on the upstream card are Ricursive's.
License
Apache 2.0, same as Ripple-65B-Preview. Cite Ricursive for the base model. This repository is the EXL3 pack only.