Ripple-65B Preview — amd-serve hybrid (IQ3_XXS / IQ4_XS mix)

A non-uniform GGUF of ricursiveAI/Ripple-65B-Preview packaged for a 32 GB AMD Radeon AI PRO R9700 by the amd-serve recipe family.

25.90 GiB · 3.43 bits per weight. Flat IQ3_XXS of the same model is 24.25 GiB at 3.22 BPW; the extra 1.65 GiB is spent where it changes output.

What is different from a flat quant

Tensor	Type	Rationale
`ffn_gate`, `ffn_up`	`iq3_xxs`	79% of parameters; the cheapest place to save
`ffn_down`	`iq4_xs`	bandwidth-critical and the most sensitive FFN tensor
`attn_k`, `attn_v`	`q4_K`, `q5_K`	small tensors carrying outsized weight
`token_embd`	`q5_K`	vocabulary fidelity
`output`	`q6_K`	untied head
`attn_q`, `attn_gate`, `attn_output`	base IQ3 mix	unchanged

Provenance

Source weights: the publisher's Q8_0 GGUF (two shards, 68.8 GB). Requantized with --allow-requantize; the BF16 release (129 GB) did not fit on the build disk.
Importance matrix: computed with llama-imatrix --parse-special over 300 chunks of the publisher's public calibration build (AtomicChat/calib-corpora, builds/ripple-65b/calib_train.txt, 5 M-token target). --parse-special matters: without it the chat markup tokenizes as punctuation and the agentic/tool slices calibrate on text the model never sees.
Importance-matrix source precision: the 3-bit preview, not Q8. A 65B model in Q8 cannot be held in VRAM alongside anything else on a 32 GB card. Disclosed because it is a weaker matrix than one computed on BF16.
Converter: the open llama.cpp pull request adding the ricursive architecture, at a pinned commit.

Requirements

This architecture is not in released llama.cpp, LM Studio or Ollama. Loading this file requires a build from the open ricursive architecture pull request. Until it merges, ordinary runners reject the file.

Measured on the target card

Radeon AI PRO R9700 (32 GB), Windows 11, Vulkan backend, 12K served context, ubatch 2048, Flash Attention unavailable for this architecture:

Measurement	Result
Decode	16.0 tokens/s
Prefill (7.4K prompt)	355 tokens/s
VRAM resident	29.3 GB
Shared-memory spill	~0.3 GB

The flat 3-bit baseline scored 77% on MMLU-Pro (2,400-token cap, 77 answers truncated), 100% on GSM8K-50 and 6/6 on a coding set on the same card. This hybrid package has not yet been scored on a held-out benchmark; it is published for reproduction and comparison, and its quality delta over the flat quant is unmeasured.

Serving notes

Serve at 10–12K context. The KV cache costs about 320 KB per token, so weights plus context exceed 32 GB beyond roughly 16K.
Do not enable Flash Attention for this architecture on Vulkan; it is not supported and falls back to the CPU.
Keep ubatch at 2048.