A fitted KV bridge: Qwen3-0.6B reads the prompt, and this bridge translates its internal state into the key/value memory Qwen3-8B would have built. The 8B, 13.6 times larger, then answers from that memory, reading one token of the prompt itself.
It applies LatentPort, Simon Villani's method (arXiv:2609.25053). The reproduction it belongs to is PrattleNot.
⚠️ Security note. When bridged, the answering model does not read the prompt itself, the system prompt included, and the bridge does not reliably carry its instructions. Measured on six of our pairs, a rule set in a system prompt is broken more often bridged than when the large model reads the prompt. Do not rely on system-prompt instructions as guardrails for a bridged model: have the large model read the system prompt itself, or enforce policy outside the model.
Held-out questions, never fitted on:
| share of the read-nothing → read-everything gap the bridge closes (NCR) | the same, from a wrong donor | correct of 7 | |
|---|---|---|---|
| with the paper's learned correction | 0.803 | 0.092 | 7, the same as the 8B alone |
NCR is the paper's native-closure ratio (arXiv:2609.25053, Eq. 8):
(ℓ_empty − ℓ_bridged) / (ℓ_empty − ℓ_native), where each ℓ is the 8B's loss on its own native
reply, scored teacher-forced. It is a loss ratio, not a rate of matching tokens. Per token, the 8B's
loss on its own reply is 0.089 reading everything, 0.451 bridged, 1.923 reading nothing, and 1.755
from a wrong question's memory. That last is the control.
The bridged 8B answers all seven questions, as the 8B reading the prompt itself does. The 8B is closer to its own reply reading everything (0.089) than bridged (0.451): a 0.6B carries most of the prompt, not all of it.
| where the two models run | time to first token, bridged 8B vs the 8B reading the prompt | the ceiling in the same run (8B alone ÷ 0.6B alone) |
|---|---|---|
| one RTX 3090, both models on it | 2.82× (2.82–2.83) sooner | 6.03× |
| two RTX 3090s, one model on each | 2.51× (2.49–2.53) sooner | 5.80× |
Both models in bf16; synthetic prompts of about 256 tokens, 8 prompts × 3 repetitions in rotated order, one bridge token, and every run's counters checked, so the bridged 8B read no prompt history itself. The figure is the geometric mean of native ÷ bridged with its 95% interval, LatentPort's own statistic. A 0.6B reads a prompt about six times faster than an 8B, so the ceiling is high. On two cards the handoff crosses between them through the host, which costs a little.
To repeat it from a PrattleNot checkout:
PRATTLE_SOURCE_DTYPE=bf16 PRATTLE_TARGET_DTYPE=bf16 PRATTLE_DEVICES=cuda:0 just speedup models/Qwen3-0.6B models/Qwen3-8B models/bridges/qwen3-0.6b-to-8b.safetensors 256
c1899de289a04d12100db370d81485cdf75e47ca and Qwen/Qwen3-8B
at b968826d9c46dd6066d109eabc6255188de91218, both Apache-2.0. 28 layers in the 0.6B and 36 in the
8B, both with 8 KV heads × 128. Hidden width is 1024 in the 0.6B and 4096 in the 8B. Prompts use
Qwen3's chat template with thinking off.qwen3-0.6b-to-8b.safetensors, 307 MB, with no __metadata__:
| tensors | what they hold |
|---|---|
k.<l>, v.<l> (f16, 2049 × 1024) |
per target layer, the map from the 0.6B's residual stream to the 8B's keys / values |
ck.<l>, cv.<l> (f32, 128 × 128) |
per target layer, the learned correction folded to one matrix per head width |
layer_of |
which 0.6B layer feeds each 8B layer |
meta |
the shape constants the reader needs |
It is read by prattle_core::bridge in PrattleNot, in Rust. It is not a transformers model, and it
contains no weights of either Qwen3 model. Fetch those from Qwen.
PrattleNot is a derivative work of LatentPort by Simon Villani: https://latentport.com/, https://github.com/LatentPort/latentport, licensed under the Apache License, Version 2.0. The bridge design, the one-token handoff and the learned correction are the paper's (arXiv:2609.25053v1). Qwen3 is by the Qwen team, under Apache-2.0. This bridge is released under Apache-2.0.
Qwen/Qwen3-0.6B