⚡ Runs on the QVB8 Inference Engine
This pack is made for the QVB8 Inference Engine. It does not load in
transformersor other engines. The engine decodes the compressed weights on the GPU, inside its own CUDA kernels, so a large model runs on small hardware. Get the engine: github.com/addiesam/qvb8 -- use v1.1.0 or later (release).
Certified L∞-bounded compression: for every restored weight, the error against the original is less than the error bound tau.
the DFlash2 draft model of Ripple-65B-Preview (speculative decoding), compressed with QVB8R (entropy-coded QVB8) at tau = 0.005. This is a weight pack for the
QVB8 inference engine; it is not loadable by transformers directly.
| Source model | ricursiveAI/Ripple-65B-Preview-DFlash2 (bf16) |
| Format | QVB8R (entropy-coded QVB8) |
| Certified for | fp16 and bf16 (restore: fp16+bf16 in manifest.json) |
| tau | 0.005 (for every restored weight, the error against the original bf16 value is less than tau) |
| Compressed tensors | 49, average 6.65 bits per weight over them |
| Size on disk | 3.4 GB |
All packs of this model. Use one target pack and one matching draft pack with the QVB8 Inference Engine.
| Pack | Format | tau | Size | Download |
|---|---|---|---|---|
| Ripple-65B-Preview | qvb8 | 1e-3 | 57 GB | Ripple-65B-Preview-QVB8-1e-3 |
| Ripple-65B-Preview | qvb8r | 1e-3 | 47 GB | Ripple-65B-Preview-QVB8R-1e-3 |
| Ripple-65B-Preview | qvb8 | 2e-3 | 49 GB | Ripple-65B-Preview-QVB8-2e-3 |
| Ripple-65B-Preview | qvb8r | 2e-3 | 38 GB | Ripple-65B-Preview-QVB8R-2e-3 |
| Ripple-65B-Preview | qvb8 | 2.5e-3 | 48 GB | Ripple-65B-Preview-QVB8-2.5e-3 |
| Ripple-65B-Preview | qvb8r | 2.5e-3 | 35.5 GB | Ripple-65B-Preview-QVB8R-2.5e-3 |
| DFlash2 draft model | qvb8 | 5e-3 | 4.1 GB | Ripple-65B-Preview-DFlash2-QVB8-5e-3 |
| DFlash2 draft model | qvb8r | 5e-3 | 3.4 GB | Ripple-65B-Preview-DFlash2-QVB8R-5e-3 |
Most quantization is judged by an average: a perplexity or benchmark score. An average hides the worst case. A format can look good on average and still leave a few weights with a large error, and a large error in the wrong weight can flip a single token. The result is a sudden wrong word or a stray character from another language in the middle of a normal answer, which an average score may not show. In our own tests with a 4-bit floating-point format we saw this in server logs.
L∞-bounded means the largest error is controlled, not the average. For this pack:
manifest.json (max_err);What the bound does not do: it covers the stored weights only. It does not promise that the model's answers are identical to the original's, or correct. Even with the exact same unquantized FP16 weights, different inference engines, batch sizes, or floating-point operation orders will still produce slightly different outputs due to non-associative GPU math. See the engine README for the full disclaimer.
Each weight matrix is packed with a per-row bit width and checked after packing: the largest error between the
restored value and the original bf16 weight is recorded per tensor in manifest.json (max_err), and no tensor
exceeds tau. The packing uses no calibration data; it depends only on the weights.
Install the engine (https://github.com/addiesam/qvb8), then follow its README. The target pack and the drafter pack are passed as MODEL and DRAFT.
2026-11-11: QVB8 Inference Engine v1.1.0; pack table aligned to all 8 packs
--pipeline cuda:0,cuda:1), a remote stage worker for several machines, KV cache offload to system RAM (--kv-window / --kv-host), the terminal chat is now a client for a running server, and prebuilt CUDA kernels are bundled for the DGX Spark (sm_121a) and the V100 (sm_70). The pack files are unchanged; only the engine is updated.2026-11-09: escape_tables.json added (QVB8 Inference Engine v1.0.1)
manifest.json. The pack files are unchanged, so only this one small file needs to be downloaded.escape_tables.json sits next to manifest.json. An fp16 run (--dtype fp16) without the file stops at start-up and tells you to download it; a bf16 run does not need it. Engine v1.0.0 ignores the file and runs as before (fp16 without the speed-up).2026-11-09: model card
First release (with QVB8 Inference Engine v1.0.0)
restore: fp16+bf16 in manifest.json); max_err per tensor is stored in manifest.json.The weights are derived from ricursiveAI/Ripple-65B-Preview-DFlash2 (Apache-2.0) by Ricursive AI; this pack keeps that license.
The engine code is Apache-2.0. No warranty is given: see the engine README for the full disclaimer.
I am an independent researcher, and I publish these packs and the engine openly. I need machines to run more tests: other GPU generations, and more models. If this is useful to you, please support the work:
Sponsor Dr. Addie Sam on GitHub
Sam, Chi Ngai. Certified L∞-Bounded Compression for AI Models, I: The Contract and the Virtual-Band Index. doi:10.5281/zenodo.23815007
Sam, Chi Ngai. Certified L∞-Bounded Compression for AI Models, II: Compressing a Language Model's Weights, Key–Value Cache and Activations. doi:10.5281/zenodo.23816044
有文件?提交种子。
提交种子