Cross-model KV mappers: Qwen3-0.6B to Qwen3-1.7B
Ridge mappers that turn the KV cache of Qwen3-0.6B into an approximate KV cache of Qwen3-1.7B, so the bigger model can skip most of its prefill. Independent implementation of arXiv:2608.03893 (NVIDIA, closed-form linear KV mapping inside a model family).
Serving code, fit pipeline and raw benchmarks: https://github.com/alesha-pro/vllm-cross-model-kv (runs on stock vLLM 0.26.0 as a KV connector, no fork).
Two artifacts, both fit on FineWeb-Edu (1024-token sequences, stride 4, lambda 0.01, RoPE-stripped K), each with fp32 and bf16 weights plus the full calibration manifest:
| mapper | calibration | held-out R²_K | R²_V | HellaSwag-1000 retention | WikiText tail-PPL ratio |
|---|---|---|---|---|---|
ridge-qwen3-0.6b-to-1.7b-k8-n500 |
500 seqs, k=8 (paper config) | 0.760 | 0.625 | 0.940 | 1.100 |
ridge-qwen3-0.6b-to-1.7b-k4-n200 |
200 seqs, k=4 | 0.731 | 0.574 | 0.953 | 1.090 |
k8-n500 has the best R² and was used for the headline latency numbers (warm TTFT 2.22x over full prefill at 32K on two RTX 3090s). k4-n200 maps about twice as fast because the GEMMs are half as wide, at near-equal downstream quality. Pick k8 for quality, k4 for mapper speed.
Usage: point mapper_path of the CrossModelKVConnector consumer at a mapper
directory. See the GitHub README for the full pair setup.
Trained and measured on 2x RTX 3090. Model pair: Qwen/Qwen3-0.6B -> Qwen/Qwen3-1.7B (same tokenizer, matched KV geometry: 8 KV heads, head_dim 128, 28 layers). These mappers are useless for any other pair.