A fitted KV bridge: SmolLM2-360M-Instruct reads the prompt, and this bridge translates its internal state into the key/value memory SmolLM2-1.7B-Instruct would have built. The 1.7B then answers from that memory, reading one token of the prompt itself.
It applies LatentPort, Simon Villani's method (arXiv:2609.25053), across a five-times size gap. The reproduction it belongs to is PrattleNot. Its smaller sibling, SmolLM2-135M → 360M, runs in the browser. This one runs on the server for now.
⚠️ Security note. When bridged, the answering model does not read the prompt itself, the system prompt included, and the bridge does not reliably carry its instructions. Measured on six of our pairs, a rule set in a system prompt is broken more often bridged than when the large model reads the prompt. Do not rely on system-prompt instructions as guardrails for a bridged model: have the large model read the system prompt itself, or enforce policy outside the model.
Held-out questions, never fitted on:
| share of the read-nothing → read-everything gap the bridge closes (NCR) | the same, from a wrong donor | correct of 7, one bridge token | |
|---|---|---|---|
| the ridge maps alone | 0.611 | −0.352 | 6, the same as the 360M alone |
| plus the paper's learned correction | 0.845 | −0.266 | 7, the same as the 1.7B reading everything |
NCR is the paper's native-closure ratio (arXiv:2609.25053, Eq. 8):
(ℓ_empty − ℓ_bridged) / (ℓ_empty − ℓ_native), where each ℓ is the 1.7B's loss on its own native
reply, scored teacher-forced. It is the share of the gap between the 1.7B reading nothing (0) and
reading the whole prompt (1) that the bridge closes. It is a loss ratio, not a rate of matching tokens.
Here the two models differ on the answer test (the 360M alone answers 6 of 7, the 1.7B 7), so it can show a gain. The correction is what lets a single bridge token carry the 1.7B's answer. The wrong-donor column is the control: the same bridge fed another question's memory, which should score far lower.
| time to first token, bridged 1.7B vs the 1.7B reading the prompt | 1.67× (1.66–1.69) sooner |
| the ceiling in the same run (1.7B alone ÷ 360M alone) | 2.59× |
Measured on one RTX 3090, with both models on the same card: synthetic prompts of about 256 tokens, 8 prompts × 3 repetitions in rotated order, one bridge token, and every run's counters checked, so the bridged 1.7B read no prompt history itself. The figure is the geometric mean of native ÷ bridged with its 95% interval, LatentPort's own statistic. The speed-up grows with prompt length: on LatentPort's own 4B → 27B pair, reproduced with upstream's harness, it is 1.36× at 128 tokens and 3.39× at about 16K. So these short prompts are the hard case.
To repeat it from a PrattleNot checkout:
PRATTLE_DEVICES=cuda:0 just speedup models/SmolLM2-360M-Instruct models/SmolLM2-1.7B-Instruct models/bridges/smollm2-360m-to-1.7b.safetensors
The method, and how to repeat every number on your own machine, are in PrattleNot's
documents/BRIDGE.md
and documents/REPRODUCTION.md §5.2.
a10cc1512eabd3dde888204e902eca88bddb4951 and
HuggingFaceTB/SmolLM2-1.7B-Instruct at
31b70e2e869a7173562077fd711b654946d38674. They have different shapes: 32 layers with 5 KV heads
against 24 layers with 32 KV heads, both 64 wide. So the maps read the 360M's residual stream
rather than its own keys and values.just bridge-fit): closed-form ridge regressions, one per target layer for keys and
one for values, read from the 360M's residual stream at two taps. Measured at about 2.5 hours.just bridge-correct): the paper's rank-4 residual correction per target layer
and role, trained through the frozen 1.7B against the native 1.7B's distribution (3 epochs,
loss 0.316 → 0.188 → 0.144). It is folded into the file as I + BUᵀ. Measured at about 1.5 hours.crates/core/src/bridge_corpus.rs.
The held-out questions in the same file are never fitted on, and a unit test keeps the two sets
disjoint.smollm2-360m-to-1.7b.safetensors, 378 MB, with no __metadata__:
| tensors | what they hold |
|---|---|
k.<l>, v.<l> (f16, 1921 × 2048) |
per target layer, the ridge map from the 360M's two residual-stream taps (plus an intercept) to the 1.7B's keys / values |
ck.<l>, cv.<l> (f32, 64 × 64) |
per target layer, the learned correction folded to one matrix per head width |
layer_of |
which 360M layer feeds each 1.7B layer |
meta |
the shape constants the reader needs |
It is read by prattle_core::bridge in PrattleNot, in Rust. It is not a transformers model, and it
contains no weights of either SmolLM2 model. Fetch those from
HuggingFaceTB.
PrattleNot is a derivative work of LatentPort by Simon Villani: https://latentport.com/, https://github.com/LatentPort/latentport, licensed under the Apache License, Version 2.0. The bridge design, the one-token handoff and the learned correction are the paper's (arXiv:2609.25053v1). SmolLM2 is by Hugging Face, under Apache-2.0. This bridge is released under Apache-2.0.
HuggingFaceTB/SmolLM2-1.7B-Instruct