qwen3.5-4b-engram

A LoRA adapter for Qwen/Qwen3.5-4B that answers questions from an AI assistant's retrieved long-term memories. It gives short answers, resolves relative dates ("last week") from the memory dates, and prefers the current value when outdated memories are also retrieved ("I live in Pune" next to "moved to Berlin").

Author: Abhash Chakraborty (M.Tech Integrated, VIT Bhopal University)
Demo: Memory models demo (real outputs) · Kaggle notebook (your own inputs, free T4) · demo/ in this repo (Gradio app)
Benchmarks: Kaggle notebook · Kaggle Benchmark
Licence: Apache 2.0 (adapter; the base model is Apache 2.0)

Results

Held-out questions, never trained or validated on. Judge accuracy uses a generous LLM judge (gpt-5-mini): the same fact in any wording counts as correct. Every model gets the same prompts, greedy decoding and thinking off.

Model	LoCoMo, judge	LoCoMo, token F1	current vs outdated, judge	latency p50 on a T4	peak GB
Qwen3.5-4B + this adapter (fp16)	0.766	0.584	0.981	1.68 s	12.1
+ this adapter, 4-bit NF4	0.753	0.568	0.981	1.75 s	8.6
Phi-4-mini-instruct (3.8B)	0.662	0.464	0.923	1.00 s	9.7
SmolLM3-3B	0.656	0.252	0.858	1.34 s	7.7
Gemma-4-E4B-it	0.643	0.451	0.935	1.00 s	14.3
Qwen3-4B-Instruct-2507	0.636	0.432	0.955	1.36 s	10.4
Qwen3.5-4B (no adapter)	0.630	0.512	0.955	1.01 s	12.0
Granite-4.0-micro (3.4B)	0.565	0.473	0.877	0.82 s	8.4
gpt-5-mini (API, reference)	0.870	0.568	0.974

The adapter beats every open model of its size on held-out LoCoMo (exact McNemar p < 0.025 in each pair), and is level with the Qwen models and ahead of the rest on current-vs-outdated questions. The 4-bit build is not measurably worse (p = 0.81) and fits an 8 GB GPU. It still trails gpt-5-mini on open-ended LoCoMo questions.

On the public Kaggle Benchmark engram memory QA (309 questions, judged by gemini-3.8-flash), the adapter's calibrated score of 0.864 is level with Claude Haiku 4.5 (0.864) and ahead of GPT-5.4 mini (0.854).

Use

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-4B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-4B", dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "abhash-chakraborty/qwen3.5-4b-engram").eval()

prompt = """You answer questions about a long conversation between the user and their assistant, using memories retrieved from it.

Memories (date in brackets):
[12 January 2025] User: I live in Pune.
[03 March 2025] User: Signed the lease, moving to Berlin next week.

Question: Where do I live? (asked on 20 March 2025)

Answer with a short phrase (a few words), not a sentence. Resolve relative times ("last week", "yesterday") into
actual dates using the memory dates. If the memories do not contain the answer, give your best guess."""
text = tok.apply_chat_template([{"role": "user", "content": prompt}], tokenize=False, add_generation_prompt=True, enable_thinking=False)
enc = tok(text, return_tensors="pt").to(model.device)
out = model.generate(**enc, max_new_tokens=32, do_sample=False)
print(tok.decode(out[0, enc.input_ids.shape[1]:], skip_special_tokens=True))   # -> Berlin

run_example.py in this repo is the same example as a script. Thinking mode must stay off: the adapter was trained without it.

Training


Adapter	LoRA r 16, α 32, dropout 0.05 on every linear layer of the language model (attention, gated-delta linear attention, MLP): 32.5M trainable parameters (0.77%)
Data	1,816 examples: public LoCoMo conversations 26, 30, 41, 42 (each question twice, with different distractors) and four simulated lives. Validation: conversation 43 and a fifth life
Stage 1: SFT	loss on answer tokens only, lr 1e-4, effective batch 8; best checkpoint after 908 examples
Stage 2: GRPO	4 samples per question (temperature 1.0), reward = LoCoMo token F1, group-normalised advantage, on-policy, lr 1e-5; best after 400 questions
Selection	validation only
Compute	about 8.5 GPU-hours on Kaggle (2 × Tesla T4), fp16 base with an fp32 adapter

training_log_sft.json and training_log_grpo.json hold the validation curves.

Requirements

fp16/bf16: a GPU with about 10 GB (12 GB with batched generation). 4-bit NF4: about 8.6 GB peak. CPU works but slowly (the demo Space: 30-60 s per answer). transformers>=5.17, peft>=0.21.

Limitations

English only, evaluated with contexts that contain the gold evidence plus distractors, not with a retriever end to end.
The GRPO reward (token F1) favours short answers. That cost some accuracy on list-type (multi-hop) questions.
Still behind gpt-5-mini on open-ended conversational questions (LoCoMo judge 0.766 vs 0.870).
The judge is one of the compared models, and each variant was trained once.

Citation

@misc{chakraborty2026qwenengram,
  author = {Abhash Chakraborty},
  title  = {qwen3.5-4b-engram: a LoRA adapter for answering from AI agent memories},
  year   = {2026},
  url    = {https://huggingface.co/abhash-chakraborty/qwen3.5-4b-engram}
}

Acknowledgements

Qwen3.5-4B by the Qwen team (Apache 2.0). LoCoMo (Maharana et al., 2024) for public conversations. Kaggle for the free T4 GPUs.

abhash-chakraborty/qwen3.5-4b-engram