A fitted KV bridge: Qwen3-0.6B reads the prompt, and this bridge translates its internal state into the key/value memory Qwen3-32B would have built. The 32B, about 54 times larger, then answers from that memory, reading one token of the prompt itself.
It applies LatentPort, Simon Villani's method (arXiv:2609.25053), across a 54-fold gap. The reproduction it belongs to is PrattleNot.
It falls short of our answer bar. Bridged, the 32B answers 6 of 7 held-out questions; the 32B reading the prompt itself answers all 7. It is published with that miss stated. See What it measures.
It is for the 32B quantized to Q4K. The bridge was fitted against Qwen3-32B with its linear layers quantized to Q4K when it loads (PrattleNot's
PRATTLE_TARGET_DTYPE=q4k), and must be used with the 32B loaded the same way.
⚠️ Security note. When bridged, the answering model does not read the prompt itself, the system prompt included, and the bridge does not reliably carry its instructions. Measured on six of our pairs, a rule set in a system prompt is broken more often bridged than when the large model reads the prompt. Do not rely on system-prompt instructions as guardrails for a bridged model: have the large model read the system prompt itself, or enforce policy outside the model.
Held-out questions, never fitted on:
| share of the read-nothing → read-everything gap the bridge closes (NCR) | the same, from a wrong donor | correct of 7 | |
|---|---|---|---|
| the maps alone | 0.794 | — | — |
| plus the paper's learned correction | 0.866 | 0.197 | 6; the 32B alone: 7, the 0.6B alone: 7 |
NCR is the paper's native-closure ratio (arXiv:2609.25053, Eq. 8):
(ℓ_empty − ℓ_bridged) / (ℓ_empty − ℓ_native), where each ℓ is the 32B's loss on its own native
reply, scored teacher-forced. It is a loss ratio, not a rate of matching tokens. Per token, the 32B's
loss on its own reply is 0.188 reading everything, 0.410 bridged, 1.844 reading nothing, and 1.518
from a wrong question's memory.
The miss, in full. Asked what gas plants breathe in, the bridged 32B answers "Plants do not 'breathe' in the same way animals do, but they perform a process called photosynthesis and also carry out cellular respiration…" and does not name carbon dioxide within the reply budget. Letting the 32B read 2, 4 or 8 bridge tokens itself instead of one does not change that answer.
| prompt | time to first token, bridged 32B vs the 32B reading the prompt | the ceiling in the same run (32B alone ÷ 0.6B alone) |
|---|---|---|
| ~256 tokens | 4.21× (4.19–4.23) sooner | 11.60× |
| ~1024 tokens | 7.17× (7.11–7.24) sooner | 14.33× |
Measured on two RTX 3090s, the 0.6B in bf16 on one and the 32B quantized to Q4K on the other: synthetic prompts, 8 prompts × 3 repetitions in rotated order, one bridge token, and every run's counters checked, so the bridged 32B read no prompt history itself. The figure is the geometric mean of native ÷ bridged with its 95% interval, LatentPort's own statistic. The speed-up grows with the prompt, because the 32B skips more reading. At ~2048 tokens the 32B reading the prompt itself runs its card out of memory in this runtime, so there is no figure there.
To repeat it from a PrattleNot checkout:
PRATTLE_SOURCE_DTYPE=bf16 PRATTLE_TARGET_DTYPE=q4k PRATTLE_DEVICES=cuda:0,cuda:1 just speedup models/Qwen3-0.6B models/Qwen3-32B models/bridges/qwen3-0.6b-to-32b.safetensors 256
Pass 1024 as the last argument for the longer prompts.
c1899de289a04d12100db370d81485cdf75e47ca and Qwen/Qwen3-32B
at 9216db5781bf21249d130ec9da846c4624c16137, both Apache-2.0. 28 layers in the 0.6B and 64 in the
32B, both with 8 KV heads × 128; hidden width 1024 and 5120. Prompts use Qwen3's chat template with
thinking off.qwen3-0.6b-to-32b.safetensors, 546 MB, with no __metadata__:
| tensors | what they hold |
|---|---|
k.<l>, v.<l> (f16, 2049 × 1024) |
per target layer, the map from the 0.6B's residual stream to the 32B's keys / values |
ck.<l>, cv.<l> (f32, 128 × 128) |
per target layer, the learned correction folded to one matrix per head width |
layer_of |
which 0.6B layer feeds each 32B layer |
meta |
the shape constants the reader needs |
It is read by prattle_core::bridge in PrattleNot, in Rust. It is not a transformers model, and it
contains no weights of either Qwen3 model. Fetch those from Qwen.
PrattleNot is a derivative work of LatentPort by Simon Villani: https://latentport.com/, https://github.com/LatentPort/latentport, licensed under the Apache License, Version 2.0. The bridge design, the one-token handoff and the learned correction are the paper's (arXiv:2609.25053v1). Qwen3 is by the Qwen team, under Apache-2.0. This bridge is released under Apache-2.0.
Qwen/Qwen3-0.6B