A fitted state bridge: Granite 4.0 H 1B reads the prompt, and this bridge translates what it built into the memory Granite 4.0 H Tiny, a 7B mixture-of-experts with 64 experts, needs. H Tiny then answers from that memory, reading one token of the prompt itself.
It applies LatentPort, Simon Villani's method (arXiv:2609.25053), to a model whose memory is mostly Mamba-2 recurrent state. The reproduction it belongs to is PrattleNot.
⚠️ Security note. When bridged, the answering model does not read the prompt itself, the system prompt included, and the bridge does not reliably carry its instructions. Measured on six of our pairs, a rule set in a system prompt is broken more often bridged than when the large model reads the prompt. Do not rely on system-prompt instructions as guardrails for a bridged model: have the large model read the system prompt itself, or enforce policy outside the model.
Both Granite models have 40 layers: 36 Mamba-2 layers and 4 attention layers (5, 15, 25, 35).
Held-out questions, never fitted on:
| share of the read-nothing → read-everything gap the bridge closes (NCR) | the same, from a wrong donor | correct of 7 | |
|---|---|---|---|
| the ridge maps alone | 0.916 | — | — |
| plus the paper's learned correction | 0.924 | −0.175 | 7, the same as each model alone |
NCR is the paper's native-closure ratio (arXiv:2609.25053, Eq. 8):
(ℓ_empty − ℓ_bridged) / (ℓ_empty − ℓ_native), where each ℓ is H Tiny's loss on its own native reply,
scored teacher-forced. It is a loss ratio, not a rate of matching tokens. Per token, H Tiny's loss on
its own reply is 0.184 reading everything, 0.310 bridged, 1.837 reading nothing, and 2.126 from a
wrong question's memory, which is worse than nothing.
All three models answer all seven questions, so the answer count shows no harm but cannot show a gain.
Two runs of the same command, ~256-token prompts, both models in bf16, 8 prompts × 3 repetitions:
| run | one RTX 3090, both models on it | two RTX 3090s, one model on each |
|---|---|---|
| 1 | 1.98× (1.87–2.10) of a 3.68× ceiling | 1.79× (1.61–1.98) of 3.93× |
| 2 | 2.06× (1.95–2.18) of 3.95× | 1.78× (1.72–1.85) of 3.99× |
The figure depends on the host CPU more than for any other bridge we publish. H Tiny routes each token to its experts on the host CPU, so a CPU kept busy by other work slows every arm and compresses the ratio: an earlier run made alongside a job using every core read 1.38× of 2.69×. Each figure is the geometric mean of native ÷ bridged with its 95% interval, LatentPort's own statistic, and every run's counters were checked, so the bridged H Tiny read no prompt history itself.
The speed-up stays well below the ceiling for a second reason too: at the handoff H Tiny must run its own projection, convolution and scan over 36 layers of translated input, which a plain key/value bridge does not have to do.
To repeat it from a PrattleNot checkout:
PRATTLE_SOURCE_DTYPE=bf16 PRATTLE_TARGET_DTYPE=bf16 PRATTLE_DEVICES=cuda:0 just speedup models/granite-4.0-h-1b models/granite-4.0-h-tiny models/bridges/granite-4.0-h-1b-to-h-tiny.safetensors 256
d18cca4c121edb87d022116d281ce212c9136f57 and
ibm-granite/granite-4.0-h-tiny at
791e0d3d28c86e106c9b6e0b4cecdee0375b6124, both Apache-2.0. Both have hidden width 1536, 12 query
heads and 4 KV heads of width 128; H Tiny routes each token to 6 of its 64 experts. Both are held in
bf16.just bridge-fit-granite): closed-form ridge regressions (λ = 10⁻³), all 40 layers
mapped, each reading the 1B's residual stream at two taps, over 43,345 rows from replies of up to
192 tokens. R² is 0.74–0.97 for the Mamba-2 inputs, 0.87–0.94 for keys and 0.74–0.87 for values.just bridge-correct-granite): the paper's rank-4 residual correction for the
attention layers' keys and values, trained through the frozen H Tiny against its own native
distribution (451 questions, 3 epochs, loss 0.081 → 0.069 → 0.062), and folded into the file as
I + BUᵀ.crates/core/src/bridge_corpus.rs.
The held-out questions are never fitted on.The method, and how to repeat every number, are in PrattleNot's
documents/BRIDGE.md
and documents/REPRODUCTION.md.
granite-4.0-h-1b-to-h-tiny.safetensors, 366 MB, with no __metadata__:
| tensors | what they hold |
|---|---|
k.<l> for a Mamba-2 layer (f16, 3073 × 1536) |
the ridge map from the 1B's two residual-stream taps (plus an intercept) to H Tiny's normed input rows for that layer |
k.<l>, v.<l> for an attention layer (f16, 3073 × 512) |
the ridge map to H Tiny's keys / values |
ck.<l>, cv.<l> (f32, 128 × 128) |
per attention layer, the learned correction folded to one matrix per head width |
layer_of |
which 1B layer feeds each H Tiny layer |
meta |
the shape constants the reader needs |
It is read by prattle_core::bridge in PrattleNot, in Rust. It is not a transformers model, and it
contains no weights of either Granite model. Fetch those from
IBM Granite.
PrattleNot is a derivative work of LatentPort by Simon Villani: https://latentport.com/, https://github.com/LatentPort/latentport, licensed under the Apache License, Version 2.0. The bridge design, the one-token handoff and the learned correction are the paper's (arXiv:2609.25053v1); translating a recurrent layer's inputs rather than its state is ours. Granite 4.0 is by IBM, under Apache-2.0. This bridge is released under Apache-2.0.
ibm-granite/granite-4.0-h-1b