kev-mem-0.8b

A System-1 decision model for an AI assistant's long-term memory. Given a new memory and its nearest existing memories, it answers typed questions in one forward pass, with calibrated probabilities:

• how the new memory relates to each existing one (duplicate / updates / elaborates / related / unrelated);
• whether each existing memory is now outdated;
• the new memory's category, kind and importance, and whether it is crucial or time-bound.

It is a LoRA adapter and decision head on Qwen/Qwen3.5-0.8B-Base, fine-tuned from jaredpalmer/kev-0.8b. It serves the same POST /v1/systemone API as TypeSafe's Jev, so any Jev client works against it unchanged.

• Author: Abhash Chakraborty (M.Tech Integrated, VIT Bhopal University)
• Demo: Memory models demo (real outputs) · Kaggle notebook (your own inputs, free T4) · demo/ in this repo (Gradio app)
• Benchmarks: Kaggle notebook · Kaggle Benchmark for hosted LLMs
• Licence: Apache 2.0

Results

Held-out test sets, never trained on. The score of each (new, old) pair is p_update = (P(relation = updates) + P(outdated)) / 2, and the old memory is replaced at p_update ≥ 0.70.

Hosted LLMs prompted with the same definition (mean AUC on stated updates over both tests): Gemini 3.7 Flash 0.892, Claude Haiku 4.5 0.889, GPT-5.4 mini 0.873; kev-mem 0.954.

Use

With Kev's server:

git clone https://github.com/jaredpalmer/kev.git && cd kev
uv sync --extra serve
uv run python -m kev.serve --run abhash-chakraborty/kev-mem-0.8b --port 8009

curl -s localhost:8009/v1/systemone -H 'content-type: application/json' -d '{
  "state": "NEW MEMORY: I moved to Berlin last week.\n\nEXISTING MEMORIES:\n[1] I live in Pune.",
  "questions": {
    "rel_1":   {"type": "choice", "instructions": "How does the NEW MEMORY relate to EXISTING MEMORY [1]? The new memory...",
                "criteria": {"duplicate": "says the same thing as the existing memory",
                             "updates": "changes the facts: the existing memory is no longer true",
                             "elaborates": "adds detail about the same thing; the existing memory stays true",
                             "related": "is about a related topic; the existing memory stays true",
                             "unrelated": "is about something else"}},
    "stale_1": {"type": "noul", "instructions": "Given the NEW MEMORY, is EXISTING MEMORY [1] now outdated or no longer true?"}
  }}'

In Python, with Kev's loader:

from kev.api import SystemOneRequest, to_answers, to_record
from kev.checkpoint import Checkpoint, LoadOptions
tok, model = Checkpoint("abhash-chakraborty/kev-mem-0.8b").load("cpu", LoadOptions(attn="sdpa", temperature=1.0))
req = SystemOneRequest(state=..., model="kev-latest", questions=...)
rec, meta = to_record(req)
answers = to_answers([p.tolist() for p in model.probs(model.encode(tok, rec))], meta)

The exact questions this model was trained on are in the demo Space (questions.json).

Training

Requirements

A GPU with 4 GB or more (about 3.9 GB measured on a T4 through Kev's server, 0.8 s per decision), or a CPU (a few seconds per decision). Python 3.12/3.13, transformers>=5.17, peft>=0.21, torch>=2.6.

Limitations

• Input style. The model was trained on facts as the memory engine stores them: third person, one fact each ("User lives in Pune."). Raw first-person chat ("I live in Pune") is out of distribution and misses more changes; the demo shows both styles side by side.

• Specialised definition. "Update" means the existing memory now describes an old value of an attribute that changed (a job, a city, a phone, the show being watched), including past events that mention the old value. The model was trained for this definition; Jev and Kev as released are general-purpose, so part of the gain is specialisation.

• English only, on personal-assistant-style memories.

• Mostly synthetic tests. The test lives are simulated; the live test uses LLM-written fact text.

• Conservative at the gate. At 0.70 it replaces 14-24% of stated changes and never a true memory; a lower, validation-chosen gate recovers more.

• Single training run, with no confidence intervals on the AUCs.

Citation

@misc{chakraborty2026kevmem,
  author = {Abhash Chakraborty},
  title  = {kev-mem-0.8b: a fine-tuned System-1 decision model for AI agent memory},
  year   = {2026},
  url    = {https://huggingface.co/abhash-chakraborty/kev-mem-0.8b}
}

Acknowledgements

Kev by Jared Palmer (Apache 2.0), which is both the starting point and the serving stack. Qwen3.5-0.8B-Base by the Qwen team (Apache 2.0). TypeSafe for Jev and the System-One API. The LoCoMo benchmark (Maharana et al., 2024) for public conversations.