CLUTRR (Sinha and others, 2019) is the field's benchmark for kinship reasoning from text. A short story states a chain of k family relations, and the task is to name the relation between the first and the last person, out of 20. Systems train on short stories and are tested on stories of up to 10 relations.
A frozen Qwen2.5-7B-Instruct reads each pair of people named close together in the story. Beside it, a block of cold attention chains what it read with a rule table it learned from the stories' answers, and answers. Nothing in the language model is trained or changed, and no rule is written by hand.
| Test accuracy, all test stories (k = 2 to 10), mean of three seeds | First split (1,048 stories) | Second split (1,146 stories) |
|---|---|---|
| Read-in trained from the answers, cost read | 64.6% (64.6 / 64.1 / 65.2) | 66.6% (66.2 / 66.8 / 66.8) |
| Zero-shot reading, nothing trained in the reader, cost read | 56.5% (56.9 / 56.9 / 55.8) | 60.6% (59.6 / 61.1 / 61.0) |
| Zero-shot reading, bit block | 34.6% | 36.9% |
| The same frozen model alone, reasoning step by step | 33.3% | 38.0% |
| The same frozen model alone, one word | 31.3% | 34.3% |
Where this sits, on the same test files. On the first split's test file, Path-of-Thoughts (arXiv 2412.17963) reports GPT-5 (medium reasoning) zero-shot at 73.2% and GPT-4o with a rule module (an answer-set solver with hand-written rules) at 67.7%; GPT-4o reasoning step by step is at 57.6%. On the second split, DSR-LM (Zhang and others, 2023; a fine-tuned reader and a differentiable logic engine) reports 60.98%. The organ sits below GPT-5 and below GPT-4o with hand-written rules, with a frozen 7B open model and no hand-written rule. This is not a best on CLUTRR.
The limit of the trained read-in: a reasoning shortcut (Marconato and others, NeurIPS 2023). Trained only from answers, the read-in does not learn to read better. It reads the relations the stories state less faithfully than the model's own reading: 77 to 81% of them (seeds and splits) right, against 89 to 90%. It uses spare relation names as codes of its own (a stated mother read as "mother-in-law", a father as "father-in-law"; no CLUTRR story states an in-law), and its rule table composes the codes into the right answer. The answers improve; the reading does not.
On the benchmark's own graphs, with no language model, the block is right on 1,039 of 1,048 test stories (first split) and 1,139 of 1,146 (second), on each of three seeds, and right on every test story with 5 to 10 relations on the first split. The rule table it learns is exactly CLUTRR's 62 rules, no more and no fewer, in all six runs.
Paper: Attention at Absolute Zero (arXiv:2611.11937). Code: github.com/NeoVand/cold-attention (the code in clutrr/).
first split: trained on stories of 2, 3 and 4 relations
| k = 2 | k = 3 | k = 4 | k = 5 | k = 6 | k = 7 | k = 8 | k = 9 | k = 10 | |
|---|---|---|---|---|---|---|---|---|---|
| Trained read-in, cost read | 95 | 75 | 54 | 68 | 63 | 62 | 62 | 63 | 56 |
| Zero-shot reading, cost read | 87 | 61 | 58 | 61 | 55 | 53 | 51 | 53 | 51 |
| Zero-shot reading, bit block | 87 | 49 | 44 | 45 | 40 | 26 | 25 | 18 | 19 |
| Graph input (no language model) | 100 | 95 | 95 | 100 | 100 | 100 | 100 | 100 | 100 |
| The frozen model alone, step by step | 63 | 33 | 34 | 37 | 33 | 30 | 33 | 27 | 30 |
second split: trained on 2 and 3
| k = 2 | k = 3 | k = 4 | k = 5 | k = 6 | k = 7 | k = 8 | k = 9 | k = 10 | |
|---|---|---|---|---|---|---|---|---|---|
| Trained read-in, cost read | 94 | 77 | 76 | 74 | 67 | 63 | 54 | 53 | 55 |
| Zero-shot reading, cost read | 89 | 68 | 69 | 68 | 66 | 54 | 47 | 54 | 48 |
| Zero-shot reading, bit block | 89 | 66 | 52 | 43 | 36 | 26 | 20 | 18 | 14 |
| Graph input (no language model) | 100 | 95 | 99 | 99 | 100 | 100 | 100 | 100 | 100 |
| The frozen model alone, step by step | 58 | 39 | 51 | 45 | 41 | 32 | 25 | 32 | 27 |
The block (graph input, and the bit block of each text stage): 2,000 updates of batch 64; updates 1 to 1,000: the rule table, on the training stories of 2 relations, with ONE loop (so the only join at the question's pair is between two stated relations); updates 1,001 to 2,000: the table frozen; the read-out (20 scales, 20 offsets) on all training stories, ceil(log2 n) + 3 loops less 0 or 1, the loss read there and 4 loops later; cross-entropy on the question's relation; Adam, learning rate 0.02 with warm-up and cosine decay in each phase, gradients clipped to norm 1; the mean of 20 snapshots, one every 50 updates from update 1,050 (the released parameters). About 40 seconds on one CPU thread.
The read-in (trained stage): first a linear map fitted to the frozen model's own zero-shot reading of the training stories' pairs (60 epochs, AdamW, learning rate 0.001, batch 4,096 pairs): readin_distilled.npz, where training starts; then 12,000 updates of 64 training stories: the read-in's probabilities are the facts; the teacher table derives each relation at the question's pair with its most probable derivation; loss: cross-entropy with the story's answer plus 0.1 x cross-entropy to the zero-shot reading; AdamW, learning rate 0.0002; the read-in kept is the best on validation (checked every 500 updates). The update kept, per seed: gen_train234_test2to10: 9500, 9500, 12000; gen_train23_test2to10: 9000, 11500, 11500.
Every choice (which reading, which thresholds, which table, how long the read-in trains) was made on validation
stories; the test stories were read once per stage. text/<split>/selection.json holds those choices and the
validation numbers they were made on.
clutrr.py the code: data loader, candidate pairs, the frozen model's reading and
states (torch, transformers), the bit block and the cost read (numpy), Organ
graph/<split>/seed<k>/ the block on the benchmark's graphs
model.safetensors W (the rule weights, 20 x 20 x 20; a rule is on when W > 0), g, bias
config.json the 62 rules, training, scores by k
reference_answers.json the answer per validation and test story
text/<split>/
readin_distilled.npz the read-in fitted to the model's own reading (U, b, mu, sd)
selection.json the validation choices
zero_shot/seed<k>/ model.safetensors (the bit block learned from the zero-shot reading; its
rules are the teacher table), config.json, reference_answers.json
trained/seed<k>/ readin.npz (U, b, mu, sd), fallback.safetensors (the bit block on the
read-in's facts), config.json, reference_answers.json
index.json every organ's test accuracy; the frozen model alone, by k
from huggingface_hub import snapshot_download
import sys
root = snapshot_download("mohsenvand/cold-attention-clutrr")
sys.path.insert(0, root)
import clutrr
rows = clutrr.load_clutrr("gen_train234_test2to10", "test") # downloads CLUTRR's own file and checks its sha256
organ = clutrr.Organ("trained", "gen_train234_test2to10", seed=0, root=root) # loads Qwen2.5-7B-Instruct on first use
got = organ.answer(rows[:16])
print([g["answer"] for g in got], [r["target_text"] for r in rows[:16]])
graph = clutrr.Organ("graph", "gen_train234_test2to10", seed=0, root=root) # no language model
print(sum(g["answer"] == r["target_text"] for g, r in zip(graph.answer(rows), rows)), "of", len(rows)) # 1,039
The reading needs the frozen model, best on a GPU in float16 (about 16 GB). The block and the cost read run on a CPU.
The released code was checked on Modal against the recorded runs:
clutrr.one_rule_per_pair), as its numbers were made: one table (second split, seed 1) has two rules for (daughter, granddaughter). With both kept, that seed answers one more test story right; the mean is unchanged at 60.6%. The trained stage uses every rule.Code and weights: Apache 2.0. The frozen model is Qwen2.5-7B-Instruct (Apache 2.0), downloaded from its own repository
at the pinned revision. The CLUTRR data is CC BY-NC 4.0 and is not part of this repository; clutrr.load_clutrr
downloads it from the dataset's own repository.
@misc{mohsenvand2026absolutezero,
title = {Attention at Absolute Zero},
author = {Mohsenvand, Mostafa},
year = {2026},
eprint = {2611.11937},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
url = {https://arxiv.org/abs/2611.11937}
}
text/gen_train234_test2to10/readin_distilled.npz 330.8 kBtext/gen_train234_test2to10/trained/seed0/readin.npz 330.8 kBtext/gen_train234_test2to10/trained/seed1/readin.npz 330.8 kBtext/gen_train234_test2to10/trained/seed2/readin.npz 330.8 kBtext/gen_train23_test2to10/readin_distilled.npz 330.8 kBtext/gen_train23_test2to10/trained/seed0/readin.npz 330.8 kBtext/gen_train23_test2to10/trained/seed1/readin.npz 330.8 kBtext/gen_train23_test2to10/trained/seed2/readin.npz 330.8 kBgraph/gen_train234_test2to10/seed0/model.safetensors 32.4 kBgraph/gen_train234_test2to10/seed1/model.safetensors 32.4 kBgraph/gen_train234_test2to10/seed2/model.safetensors 32.4 kBgraph/gen_train23_test2to10/seed0/model.safetensors 32.4 kBgraph/gen_train23_test2to10/seed1/model.safetensors 32.4 kBgraph/gen_train23_test2to10/seed2/model.safetensors 32.4 kBtext/gen_train234_test2to10/trained/seed0/fallback.safetensors 32.4 kBtext/gen_train234_test2to10/trained/seed1/fallback.safetensors 32.4 kBtext/gen_train234_test2to10/trained/seed2/fallback.safetensors 32.4 kBtext/gen_train234_test2to10/zero_shot/seed0/model.safetensors 32.4 kBtext/gen_train234_test2to10/zero_shot/seed1/model.safetensors 32.4 kBtext/gen_train234_test2to10/zero_shot/seed2/model.safetensors 32.4 kBtext/gen_train23_test2to10/trained/seed0/fallback.safetensors 32.4 kBtext/gen_train23_test2to10/trained/seed1/fallback.safetensors 32.4 kBtext/gen_train23_test2to10/trained/seed2/fallback.safetensors 32.4 kBtext/gen_train23_test2to10/zero_shot/seed0/model.safetensors 32.4 kBtext/gen_train23_test2to10/zero_shot/seed1/model.safetensors 32.4 kBtext/gen_train23_test2to10/zero_shot/seed2/model.safetensors 32.4 kB