Adapters from the project Wrong Names in Natural Language Autoencoder Explanations (code and report: https://github.com/tanvincible/nla-wrong-names).
A natural language autoencoder (NLA) explains an activation of a language model by writing a paragraph about
it. The released verbalizer for Qwen2.5-7B (kitft/nla-qwen2.5-7b-L20-av) sometimes names an entity that was
never in the text. Each adapter here is that verbalizer fine-tuned for three rounds of "sample 8 explanations,
keep the best one under a score, train on it", with a different score. They exist to test what the training
reward does to wrong names, not as improved verbalizers.
| Folder | Score used to pick the kept sample | Wrong names per explanation (held-out) | True-name rate |
|---|---|---|---|
recon |
reconstruction cosine | -1.3% (not distinguishable from 0) | unchanged |
lineup |
reconstruction must pick the activation out of 4 near-twins | -0.5% (not distinguishable from 0) | unchanged |
holdback |
cosine - 0.01 per name the other samples do not support | -8.4% | unchanged |
holdback-seed1 |
same as holdback, from a different seed |
-5.1% | unchanged |
holdback-5epochs |
same, one round with 5 passes | -6.5% | unchanged |
recall |
hold back + 0.05 for naming the true entity | -2.9% (interval touches 0) | unchanged |
Changes are relative to the released verbalizer on 540 held-out Wikipedia passages, paired per entity.
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("kitft/nla-qwen2.5-7b-L20-av", torch_dtype="auto")
model = PeftModel.from_pretrained(base, "tanvincible/nla-qwen2.5-7b-L20-av-specifics-lora", subfolder="holdback")
The verbalizer takes an activation injected into its prompt, not plain text. evaluate.py in the GitHub
repository shows the full path from an activation to an explanation with an adapter applied.
LoRA rank 16 on the attention projections, learning rate 1e-4, 2,000 training activations per run.
holdback-5epochs/adapter_model.safetensors 40 MBholdback-seed1/adapter_model.safetensors 40 MBholdback/adapter_model.safetensors 40 MBlineup/adapter_model.safetensors 40 MBrecall/adapter_model.safetensors 40 MBrecon/adapter_model.safetensors 40 MBkitft/nla-qwen2.5-7b-L20-av