A fitted KV bridge: Qwen3-4B reads the prompt, and this bridge translates its internal state into the key/value memory Qwen3-8B would have built. The 8B then answers from that memory, reading one token of the prompt itself.
It applies LatentPort, Simon Villani's method (arXiv:2609.25053), to the Qwen3 family at a larger size. The reproduction it belongs to is PrattleNot.
⚠️ Security note. When bridged, the answering model does not read the prompt itself, the system prompt included, and the bridge does not reliably carry its instructions. Measured on six of our pairs, a rule set in a system prompt is broken more often bridged than when the large model reads the prompt. Do not rely on system-prompt instructions as guardrails for a bridged model: have the large model read the system prompt itself, or enforce policy outside the model.
Held-out questions, never fitted on:
| share of the read-nothing → read-everything gap the bridge closes (NCR) | the same, from a wrong donor | correct of 7 | |
|---|---|---|---|
| with the paper's learned correction | 0.890 | −0.026 | 7, the same as each model alone |
NCR is the paper's native-closure ratio (arXiv:2609.25053, Eq. 8):
(ℓ_empty − ℓ_bridged) / (ℓ_empty − ℓ_native), where each ℓ is the 8B's loss on its own native
reply, scored teacher-forced. It is a loss ratio, not a rate of matching tokens. Per token, the 8B's
loss on its own reply is 0.089 reading everything, 0.291 bridged, 1.923 reading nothing, and 1.970
from a wrong question's memory. That last is the control: another question's memory is no better
than none.
All three models answer all seven questions, so the answer count shows no harm but cannot show a gain. One answer, in full: asked what gas plants breathe in, the bridged 8B opens "Plants do not breathe in the same way animals do…" and names carbon dioxide in the next sentence. The 8B reading the prompt itself names it at once. An earlier fit, on shorter replies, began the same way and was graded wrong on that question. Fitting on longer replies (192 tokens, so more rows of the question's own words) is what brought the answer back.
| time to first token, bridged 8B vs the 8B reading the prompt | 1.12× (1.12–1.12) sooner |
| the ceiling in the same run (8B alone ÷ 4B alone) | 1.61× |
Measured on two RTX 3090s, the 4B on one and the 8B on the other, both in bf16: synthetic prompts of about 256 tokens, 8 prompts × 3 repetitions in rotated order, one bridge token, and every run's counters checked, so the bridged 8B read no prompt history itself. The figure is the geometric mean of native ÷ bridged with its 95% interval, LatentPort's own statistic. The two models are close in cost on these cards, so the ceiling is low, and at ~256 tokens the bridge reaches a fifth of it.
To repeat it from a PrattleNot checkout:
PRATTLE_SOURCE_DTYPE=bf16 PRATTLE_TARGET_DTYPE=bf16 PRATTLE_DEVICES=cuda:0,cuda:1 just speedup models/Qwen3-4B models/Qwen3-8B models/bridges/qwen3-4b-to-8b.safetensors 256
1cfa9a7208912126459214e8b04321603b3df60c and Qwen/Qwen3-8B
at b968826d9c46dd6066d109eabc6255188de91218, both Apache-2.0. Both have 36 layers and 8 KV heads
× 128. Hidden width is 2560 in the 4B and 4096 in the 8B. Both are held in bf16. Prompts use
Qwen3's chat template with thinking off.just bridge-fit-qwen3-8b): closed-form ridge regressions (λ = 10⁻³), one per target
layer for keys and one for values. Each reads the 4B's residual stream at two taps, over 46,319
rows from replies of up to 192 tokens. R² is 0.77–0.98 for keys and 0.64–0.95 for values.just bridge-correct-qwen3-8b): the paper's rank-4 residual correction per
target layer and role. It is trained through the frozen 8B against the native 8B's
distribution (451 questions, 3 epochs, loss 0.082 → 0.052 → 0.044), and folded into the file as
I + BUᵀ.crates/core/src/bridge_corpus.rs.
The held-out questions are never fitted on.The method, and how to repeat every number, are in PrattleNot's
documents/BRIDGE.md
and documents/REPRODUCTION.md.
qwen3-4b-to-8b.safetensors, 760 MB, with no __metadata__:
| tensors | what they hold |
|---|---|
k.<l>, v.<l> (f16, 5121 × 1024) |
per target layer, the ridge map from the 4B's two residual-stream taps (plus an intercept) to the 8B's keys / values |
ck.<l>, cv.<l> (f32, 128 × 128) |
per target layer, the learned correction folded to one matrix per head width |
layer_of |
which 4B layer feeds each 8B layer |
meta |
the shape constants the reader needs |
It is read by prattle_core::bridge in PrattleNot, in Rust. It is not a transformers model, and it
contains no weights of either Qwen3 model. Fetch those from Qwen.
PrattleNot is a derivative work of LatentPort by Simon Villani: https://latentport.com/, https://github.com/LatentPort/latentport, licensed under the Apache License, Version 2.0. The bridge design, the one-token handoff and the learned correction are the paper's (arXiv:2609.25053v1). Qwen3 is by the Qwen team, under Apache-2.0. This bridge is released under Apache-2.0.
Qwen/Qwen3-4B