Long-Context Memory Curation Management

Splinter Memory — a managed-decay context curation architecture for long-horizon LLM conversations.

This card is the Hugging Face edition of the Splinter Memory white paper. It is written for the people who build LLM applications and want the context window to stay useful as the conversation grows: the memory layer, the mechanisms, the measured results, and the code that implements them.

The canonical paper lives with the code (sky-is-green/hive/tree/main/splinter); this edition adds the encoder measurement taken on the harness's own traffic (§7) and the operational notes a practitioner needs.

---

The problem, in one paragraph

A long conversation is not a long document. Facts that mattered twenty turns ago fall out of a rolling context window; the model answers from what survives, not from what happened; and the surviving context is mostly noise by then. Truncation and recency are cheap and wrong: they throw away exactly the old fact the user came back for. Keeping everything is worse - the prompt grows, the KV cache grows, the cost grows, and the model "loses the middle" while the interesting fact sits somewhere in the middle.

What this does

Splinter Memory keeps a managed memory outside the model and hands the model a bounded, relevance-ranked context every turn:

• a small fleet of bidirectional encoders ("drones") scores every stored chunk against the current query - cheap, and far better at similarity than a generative model of the same size;
• a membrane drops what is said twice and notices when the topic changes;
• a retention layer makes forgetting a design parameter - an escalating-friction decay, a remembrance pass that rescues facts worth keeping, and a "comb" (an SSD tier) that resurrects topics that returned after being evicted;
• a focal layer assembles a bounded context on an adaptive token budget and hands it to any OpenAI-compatible backend (llama.cpp, LM Studio, vLLM, a hosted API).

The model never sees the raw history. It sees the part that is worth its attention, every turn, at a cost that does not grow with the conversation.

The separation is the point. The generative model is good at producing answers and the wrong tool for selecting context; measured live, using the 27B generative model as an encoder proxy cost tens of seconds per turn on the hot path, while the 17M-parameter drone did the same job in milliseconds.

The measured results

Full protocol, per-hypothesis logic chains and falsification conditions: §5 of the canonical paper.

The whole protocol, refreshed on the 35B-A3B

The P1-P11 protocol re-run live against the 35B-A3B ternary (a 12-conversation fixture subset), to ask whether the verdict set is a property of the architecture or of the generator it was measured on:

PASS 6/11, 2 skipped - the published verdict set, exactly. The published run (2026-08-23/24, on a 27B, on the full corpus) reported P1/P3/P4/P8/P9/P11 PASS, P2/P6/P10 FAIL, P5/P7 SKIP. The same set reproduces on a different model and a smaller corpus, which says the protocol measures the architecture rather than the generator.

One of the failures now has a candidate mechanism. P10 (the drift reset) measures a detector whose threshold was calibrated for a different encoder (§7): forced on, it fires on most turns, so "the reset has no effect" may be measuring a detector that fires far too often rather than a reset that does not work. (The exact rate was measured on the pre-correction corpus and is withdrawn with the rest of §7.) The verdict stands as measured; the mechanism is now in question, which is a different claim.

And the one result that does not carry over is P3's margin. On the 27B the curated context won 115 turns outright to 26; on the 35B it is a tie (§7.3). The verdict holds (the curated context is sufficient, and at least as good on 88.4% of the turns); the margin does not. The curation's accuracy advantage is a function of the generator - what survives for a strong one is the bounded context and the cost.

Use it

Three ways, in increasing order of control:

1. As a sidecar - the Studio serves an OpenAI-compatible /v1/chat/completions that curates before it generates. Point your client at it; nothing else changes.

git clone https://github.com/sky-is-green/hive
cd hive
python -m pip install -e splinter
python -m harness --port 8765

2. As a library - the memory system is importable, engine-agnostic and has no GPU requirement for the encoder tier.

from cortex.splinter import Splinter
from cortex.config import SplinterConfig

s = Splinter(config=SplinterConfig(max_context=8192))
result = s.process_turn("what did we decide about the KV cache?", "my-conversation")
print(result.assembled.content)     # the curated context, bounded

3. As the benchmark - the P1-P12 protocol, the deterministic diagnostics and the self-contained run bundles are in the repository, so the claims can be re-measured on your own hardware and your own backend.

How it works

Per turn: user input → Cortex classifies it → Sieve scores every stored chunk → Membrane deduplicates and checks the topic → Retention applies decay and the remembrance pass → Focal assembles a bounded context → the backend generates → the reply is observed back into the store.

The four postulates

1. Context curation - for a fixed token budget, a relevance-ranked selection from the full history dominates a contiguous window of the same size, and the gap widens with conversation length.
2. Managed decay - forgetting is a design parameter, not a failure mode; escalating friction beats both unbounded growth and uniform forgetting.
3. Separation - bidirectional encoders for comprehension, an autoregressive decoder for generation, is the better allocation of the same compute.
4. Ground-truth bootstrap - an LLM auditor asked about context utilization (not answer correctness) agrees with human labels 90.25% of the time, which is enough to drive parameter tuning without a human labelling pass.

The harness that runs it, and the pipeline that manages it

The memory system is a library; the HiveBench Studio is the harness that runs it, and it is where the management pipeline lives.

The management pipeline

A long-horizon memory has to be managed: every knob that decides what the model sees used to live in three different menus, and a wrong setting was invisible. The Studio has one place for all of it - the Global harness controls - and every block of it is a baseline that the more specific thing beats:

The point is not tidiness. It is that a setting which is not consulted is indistinguishable from a setting which does not work, and the only way to know which one you have is to wire them and then measure.

The harness itself

Answer quality reporting and API escalation

The curation's effect is not only felt, it is reported. Every cascade run writes a per-item report (the verdict, the detector findings, the repair attempt, the ladder's resolution), a provenance manifest (the model sha256, the stack, the git commit, the task-set sha, the decode settings) and a generated model card; and the escalation lane harvests every ladder-resolved item to escalation.jsonl next to the report - training-side only, and read by nothing in the runtime.

The escalation ladder itself is the last stage: an answer the local verifiers still flag can go to a human gate (verbose on the first trigger, compact afterwards) and then to an API tier, with the served answer, the cost and the reason recorded. The companion paper (The Hive Cascade) documents the loop, the gold-free runtime contract and the measured verdicts; the benchmark and the run reports are published next to this card.

The evidence, hypothesis by hypothesis

The failures are kept in the paper with their evidence. A pre-registered protocol that only reports passes is not evidence.

7. The encoder: what we measured on our own traffic

Correction (2026-10-21). The corpus behind this section included an ingested knowledge-base attachment: 17 of its 82 files were chunked [block n/m] documents (5,531 of 6,977 chunks), and one conversation was loaded from two state directories. The drift figures below (the 0.945 AUC and the "fires on 571 of 628 turns" mechanism) are withdrawn, and the tuned encoder was rejected by the replay (see below). The shipped encoder and the shipped 0.92 stay. The correction and the clean measurement are in AB-RESULTS.md §2C-septies; the retrieval results elsewhere in this card come from different runs and are unaffected.

The harness gives its encoder three jobs, and only one of them is a ranking:

The prior encoder comparison measured rank only, on the synthetic fixtures, and found that six encoders (a 22M MiniLM, a 400MB cross-encoder, a 568M retrieval-specialised encoder and two contrastively tuned variants) all landed on the same top-K curve - so it concluded the ceiling was data-structural and that the remedy was "stop looking for a better encoder".

We re-measured all three jobs on the harness's own traffic - 81 conversations, 6,975 stored chunks, 4,056 duplicate pairs and 6,533 topic-change cases, labelled lexically so the label is independent of the encoder being measured (experiments/encoder_suite.py). The result changes the picture, and it is not about the ranking:

The thresholds that ship with the harness were calibrated for a different encoder. The production drone was swapped from all-MiniLM-L6-v2 to paraphrase-MiniLM-L3-v2 on 2026-08-23; the membrane's two thresholds were not re-derived.

7.1 The F1 is the wrong target, and the threshold is not the lever

The obvious conclusion - "move the thresholds to the encoder's optimum" - is falsified by a deterministic replay of the corpus through the real pipeline (50 conversations, 493 measurable turns, the P2 diagnostic's own notion of a fact; experiments/threshold_ab.py):

The duplicate test's two errors are not interchangeable. A duplicate test that fires wrongly deletes a chunk, and the fact inside it is gone irreversibly; a duplicate test that fails to fire only leaves a redundant chunk in the context, which costs budget and nothing else. F1 weights them equally, so F1 points at the wrong threshold: at 0.57 the encoder merges chunks that are about the same thing but carry different facts, and the fact recall falls by a fifth. The shipped 0.92 is conservative for a reason - its precision is 1.00, so it never deletes a fact - at the price of catching only 5% of the redundancy.

The topic-drift test is the opposite case: it only penalises the off-topic chunks, it never deletes them, so its recalibration is free.

The model is not the lever either. A small encoder tuned on the harness's own traffic (17M parameters, one epoch, the dedup task only) raises the dedup decision at 0.92 from a recall of 0.046 to 0.986 on the held-out job test. The replay then prices that decision: at the shipped 0.92 the tuned encoder keeps 32.66% of the facts against the stock's 60.51%, for a context 2.7x smaller. The decision improved; the curation nearly halved. The shipped encoder and the shipped 0.92 stay.

The reason is structural: a true duplicate merged is a fact deleted from the context, so improving the dedup decision is improving the deletion. The threshold and the model are two dials on the same trade-off curve, and the fact-preservation constraint fixes where on it the system sits. The clean replay and the three failed fine-tunes are in AB-RESULTS.md §2C-quater through §2C-septies.

7.4 The curation's cost grows with the conversation it curates

The stability run - 2,000 turns of a growing conversation, the real drone, no model (experiments.stability_500.py):

The context is flat and the cost is not. The assembled context moves by 10 tokens across a 7.6x range of store - the curation's promise holds ✓✓ - and nothing is OOM-killed over 2,000 turns ✓. But the curation's per-turn cost is linear in the store: about 7x for 7.6x the chunks, because every chunk is scored every turn.

So the memory's trade is explicit, and the published "≈18 ms added per turn" is the small-store number: the prompt stays bounded (the expensive part for the model) while the curation gets more expensive as the conversation grows. The two costs trade against each other, and the crossover - the point where curating costs more than the tokens it saves

• is a measurement the system does not yet make.

7.3 The curation's advantage is generator-dependent

The paired A/B - the same model, the same turns, the curated context against the last-4k-token window of the same history - is the memory system's headline measurement (P3). It was measured on a 27B, and it has now been measured on the 35B-A3B ternary (180 measurable turns):

On the 27B the curated arm won 115 turns outright to 26 (4.4:1). On the 35B it is 13 to 17 - a tie - and the answers are equal (81.1% against 81.7%). The "neither" bucket collapses too (145 of 175 on the 27B, 34 of 180 here): the 35B answers from whatever context it is given.

The scoping result: the better the generator is at long context, the less the curation has to add. The FIFO window is the most recent turns verbatim, and a model that uses them well does not need a curation to find them. This is what Postulate 3 (the separation) implies - the curation is a substitute for a weaker reader - and it is the first thing the measurements had not tested.

What survives. The curated context is still sufficient on 97.5% of the turns, its fidelity is ahead, and - the part the numbers above do not touch - the curation is what keeps the context bounded: the FIFO arm is 4k tokens of pure recency, and the two arms are equal at that budget. The honest claim for a strong generator is the bounded context and the cost, not the better answers.

7.2 What we changed, and what we did not

• The shipped configuration stays: the stock encoder at the shipped thresholds, because that is the configuration the published evidence was produced with.
• The next fine-tune weights a false merge above a missed merge (precision-weighted at the operating point) instead of optimising F1 - the asymmetry makes F1 the wrong target - and is measured on a held-out split, because the current one was trained and measured on the same traffic (a fit, not a generalisation).
• The full sweep, the fine-tune runs and the replay are in docs/ENCODER-NOTES.md in splinter/.

Limits, honestly

• Precision is the open question. The context that reaches the model is fatter than it needs to be. The recall evidence says the facts the model stated are in there; an independent, non-circular precision number is still open.
• The auditor sees the answer. P7's agreement check mitigates the circularity but does not remove it: the auditor is the same model family that produced the answer.
• Synthetic-corpus bias. The long-horizon corpus is generated. The measurement in §7 is on real traffic; the P1-P12 verdicts mostly are not.
• Cross-conversation contamination and hedge-replies are harness-level failure modes that look like architectural failures if you do not isolate conversations per store and filter refusals out of the store.
• The decay curve is an engineering heuristic with cognitive-science inspiration, not a validated model of human forgetting.
• The large tier is a state, not a model. The 176B-A6B answers at 60-73 tok/s warm and 3 tok/s cold, on the same engine and the same request: a fresh prompt read costs 5 tok/s and a reused prefix 71, and the cold phase is the OS pulling 60 GB of expert weights off the NVMe (the file is 20.8% page-cached on a 30 GB box). Warm and alone it answers the cascade's 28 tasks in 26.9 s an item; co-sited with the small tier, in 229.7 s. A deployment that measures the large tier cold, or with another engine resident, measures the box and not the model. Two levers pay and the rest do not: keep it warm, and run it alone. A whole-file prefetch does not - a 13 GB page cache cannot hold a 60 GB file, and the kernel's WILLNEED left it at 0% cached for 80 s. The engine's resident tier (--resident-budget-gib) does: it keeps what the VRAM cache cannot hold in ordinary RAM, and the storage read on the second request falls from 1.4 GB to 146 MB.

Reproduce

# the offline suite (deterministic diagnostics, no model needed)
pytest tests/unit tests/integration -q

# the encoder sweep on your own traffic
python -m experiments.encoder_suite --encoders l3 l6 bge-small e5-small gte-small

# the threshold replay (deterministic: no model, no generation)
python -m experiments.threshold_ab

# the cascade pilot (local model, deterministic verifiers)
open http://127.0.0.1:8765/cascade

The repository ships the run bundles, the event logs and the label files, so a verdict can be traced back to the bytes that produced it.

The artifacts

The releases are private while the results are being refreshed; they open when the refreshed numbers are in.

License and citation

MIT. See LICENSE and CITATION.cff in this repository.

@misc{splinter-memory-2026,
  title  = {Splinter Memory: A Managed-Decay Context Curation Architecture
            for Long-Horizon LLM Conversations},
  author = {sky-is-green},
  year   = {2026},
  url    = {https://github.com/sky-is-green/hive/tree/main/splinter}
}

Community invitation. The protocol is written to be challenged: every prediction has a logic chain, a measurement and a falsification condition. If your hardware produces a different verdict, that is a result - open an issue with the run bundle.