A fitted KV bridge: Qwen3.5-4B-Base reads the prompt, and this bridge translates its internal state into the memory Qwen3.5-9B-Base would have built. The 9B then answers from that memory, reading one token of the prompt itself.
This is LatentPort's own pair: Simon Villani's paper (arXiv:2609.25053) fits its bridge from Qwen3.5-4B to Qwen3.5-9B. This is our independent re-fit of that pair with the paper's method, in Rust, as part of the reproduction PrattleNot. Fidelity is the point of this bridge. Its speed-up is small today (see below).
⚠️ Security note. When bridged, the answering model does not read the prompt itself, the system prompt included, and the bridge does not reliably carry its instructions. Measured on six of our pairs, a rule set in a system prompt is broken more often bridged than when the large model reads the prompt. Do not rely on system-prompt instructions as guardrails for a bridged model: have the large model read the system prompt itself, or enforce policy outside the model.
Qwen3.5 interleaves three linear-attention (Gated DeltaNet) layers with every full-attention layer, and the bridge treats the two kinds of memory differently:
Held-out questions, never fitted on:
| share of the read-nothing → read-everything gap the bridge closes (NCR) | the same, from a wrong donor | correct of 7 | |
|---|---|---|---|
| the maps alone | 0.974 | — | — |
| plus the paper's learned correction | 0.989 | 0.308 | 7, the same as each model alone |
NCR is the paper's native-closure ratio (arXiv:2609.25053, Eq. 8):
(ℓ_empty − ℓ_bridged) / (ℓ_empty − ℓ_native), where each ℓ is the 9B's loss on its own native
reply, scored teacher-forced. It is a loss ratio, not a rate of matching tokens. Per token, the 9B's
loss on its own reply is 0.268 reading everything, 0.286 bridged with the correction, and 1.840
reading nothing.
Read the wrong-donor column with care. Fed another question's memory, the same bridge still closes 0.308 of the gap (loss 1.357), far more than our other bridges' controls. This is expected here: 24 of the 32 layers carry the 4B's own recurrent state, which holds a lot of general context whichever prompt produced it. The right donor closes 0.989, so the prompt's own content is what the 9B reads.
Both models answer all seven questions on their own. These are base models, not instruction-tuned. So the answer count shows the bridge does no harm, but it cannot show a gain.
| prompt | time to first token, bridged 9B vs the 9B reading the prompt | the ceiling in the same run (9B alone ÷ 4B alone) |
|---|---|---|
| ~256 tokens | 1.16× (1.16–1.16) sooner | 1.38× |
| ~2048 tokens | 1.15× (1.14–1.15) sooner | 1.26× |
Measured on two RTX 3090s, the 4B on one and the 9B on the other, both in bf16: synthetic prompts, 8 prompts × 3 repetitions in rotated order, one bridge token, and every run's counters checked, so the bridged 9B read no prompt history itself. The figure is the geometric mean of native ÷ bridged with its 95% interval, LatentPort's own statistic.
The ceiling is low, and that is the limit. The 4B is not much faster than the 9B on these cards: both spend about 52 ms of each first token in the Gated DeltaNet core, whose geometry they share. So even a perfect handoff could only reach 1.38×. The bridge reaches most of what is available, but it is not much. The paper itself claims no speed-up for this pair.
To repeat it from a PrattleNot checkout:
PRATTLE_SOURCE_DTYPE=bf16 PRATTLE_TARGET_DTYPE=bf16 PRATTLE_DEVICES=cuda:0,cuda:1 just speedup models/Qwen3.5-4B-Base models/Qwen3.5-9B-Base models/bridges/qwen3.5-4b-to-9b.safetensors 256
Pass 2048 as the last argument for the longer prompts.
1001bb4d826a52d1f399e183466143f4da7b741b and
Qwen/Qwen3.5-9B-Base at
68c46c4b3498877f3ef123c856ecfde50c39f404, both Apache-2.0. Both have 32 layers and 4 KV heads
of width 256. Hidden width is 2560 in the 4B and 4096 in the 9B. Both are held in bf16, and only
the language model is used.qwen3.5-4b-to-9b.safetensors, 172 MB, with no __metadata__:
| tensors | what they hold |
|---|---|
k.<l>, v.<l> (f16, 5121 × 1024) |
per full-attention layer, the map from the 4B's residual stream to the 9B's keys / values |
ck.<l>, cv.<l> (f32, 256 × 256) |
per full-attention layer, the learned correction folded to one matrix per head width |
layer_of |
which 4B layer feeds each 9B layer |
meta |
the shape constants the reader needs |
It is read by prattle_core::bridge in PrattleNot, in Rust. It is not a transformers model, and it
contains no weights of either Qwen3.5 model. Fetch those from Qwen.
PrattleNot is a derivative work of LatentPort by Simon Villani: https://latentport.com/, https://github.com/LatentPort/latentport, licensed under the Apache License, Version 2.0. The bridge design, the choice of this pair, the one-token handoff and the learned correction are the paper's (arXiv:2609.25053v1). Qwen3.5 is by the Qwen team, under Apache-2.0. This bridge is released under Apache-2.0.
Qwen/Qwen3.5-4B-Base