MemTensor / MemT-1.0-Nano
ASSEMBLED BY: UnstableLlama Information exl3 builds of the MemT-1.0-Nano ⚠️ PERSISTENT MEMORY MODEL ⚠️ that works with TabbyAPI. ExLlamaV3 fork exl3-qlora required.MemT-1.0-Nano is Qwen3.6-35B-A3B plus a trained memory block on its ten full-attention layers: the model files earlier conversation turns into a 40 MiB latent memory and answers from it instead of re-reading the whole chat.
Each revision is the stock backbone quant from yeasah/Qwen3.6-35B-A3B-exl3 (byte for byte, credit yeasah) plus the MemT files:
memt_block.safetensors (bf16, 0.84 GiB), the MemT configs, and the release chat template with a small
addition for client-side memory control. No requant.
Demo
MemT-1.0-Nano at 6 bpw on two RTX 3090s. Watch the cache and prompt history get wiped after the
/commit
(the prompt drops from 1858 tokens to 21), yet the model still remembers the story when asked the next question.
Repo Data
| REVISION | GiB | KL DIV | PPL | BACKBONE QUANT |
|---|---|---|---|---|
| 3.00bpw | 15.67 | 0.0215 | 1.3389 | 3.00bpw-H5 |
| 4.00bpw | 19.64 | 0.0069 | 1.3207 | 4.00bpw |
| 5.00bpw | 23.55 | 0.0030 | 1.3150 | 5.00bpw |
| 6.00bpw | 27.47 | 0.0018 | 1.3116 | 6.00bpw |
| bf16 | 63.61 | 0.0000 | 1.3126 | - |
./start.sh once so it sets up its venv, then install the fork into that venv (the script only installs
dependencies on its first run, so it won't overwrite the fork). Launch on a revision. Memory is on by default for this model.
EXL3_MEMT_SNAPSHOT_DIR=./memt_snapshots ./start.sh --model-name MemT-1.0-Nano-exl3-3.00bpw --max-seq-len 32768
Chat normally. Type
/commit to memorize the conversation so far (the prompt the model reads shrinks to what came
after), or /commit every 3 to let it memorize on its own. Edits roll memory back, regenerations don't memorize
twice, chats don't share memory. Every setting, the request-level controls and the test scripts are in the fork's
doc/MEMT_README.md.
Not yet measured: whether memorizing replies as the model generated them (reasoning included, the original MemT runtime's way,
EXL3_MEMT_COMMIT_GENERATED=1) recalls better than memorizing the client's
re-sent copy (the default). Both are supported; nobody has compared them.
CLI Download
Run from the tabbyAPI folder, so the model lands in Tabby's models folder.
hf download UnstableLlama/MemT-1.0-Nano-exl3 --revision "3.00bpw" --local-dir models/MemT-1.0-Nano-exl3-3.00bpw