neuronpedia/jacobian-lens

This folder contains pre-fitted Jacobian Lenses and J++ Lenses for specific models.

Jacobian Lens and J++ Lens can be explored on Neuronpedia at https://neuronpedia.org/jlens.

Jacobian Lens

The Jacobian Lenses (<model>/jlens/) were trained using Anthropic's Jacobian Lens library. The same library can be used to load the lens and local visualizer.

J++ Lens

The J++ Lenses (<model>/jpp/) are from J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses (Ayonrinde & Lindsey, 2026). J++ Lens is an improved, drop-in replacement for the Jacobian Lens. It is more coherent on earlier layers, where the Jacobian Lens is often unreliable.

Credit for the method and for the lenses marked "Ayonrinde & Lindsey" goes to the authors. Their original lens files are at koayon/jpp-lenses. We changed those files to the format that the Neuronpedia inference server and the Jacobian Lens files use. We cast the maps from fp32 to fp16 (relative error 2.7e-4 or less; 1.1e-3 for Gemma 4 31B, which has very small values in some layers). We did not change them in other ways. We fitted the lenses marked "Neuronpedia" with the paper's code and recipe.

Model	File	Fitted by	Recall@10
Qwen/Qwen3.6-27B	`qwen3.6-27b/jpp/Salesforce-wikitext/Qwen3.6-27B_jpp_lens.pt`	Ayonrinde & Lindsey	55.2%
Qwen/Qwen3.5-9B	`qwen3.5-9b/jpp/Salesforce-wikitext/Qwen3.5-9B_jpp_lens.pt`	Ayonrinde & Lindsey	55.0%
google/gemma-4-31B	`gemma-4-31b/jpp/Salesforce-wikitext/gemma-4-31B_jpp_lens.pt`	Ayonrinde & Lindsey	58%*
allenai/Olmo-3-1125-32B	`olmo-3-1125-32b/jpp/Salesforce-wikitext/Olmo-3-1125-32B_jpp_lens.pt`	Ayonrinde & Lindsey	50.0%
Qwen/Qwen3.5-122B-A10B	`qwen3.5-122b-a10b/jpp/Salesforce-wikitext/Qwen3.5-122B-A10B_jpp_lens.pt`	Ayonrinde & Lindsey	48.4%
deepseek-ai/DeepSeek-V4-Flash	`deepseek-v4-flash/jpp/Salesforce-wikitext/DeepSeek-V4-Flash_jpp_lens.pt`	Ayonrinde & Lindsey	61.4%
Qwen/Qwen3.5-0.8B	`qwen3.5-0.8b/jpp/Salesforce-wikitext/Qwen3.5-0.8B_jpp_lens.pt`	Neuronpedia	44.9%
Qwen/Qwen3.5-2B	`qwen3.5-2b/jpp/Salesforce-wikitext/Qwen3.5-2B_jpp_lens.pt`	Neuronpedia	51.5%
Qwen/Qwen3-4B	`qwen3-4b/jpp/Salesforce-wikitext/Qwen3-4B_jpp_lens.pt`	Neuronpedia	53.8%

Recall@10 is macro recall on the paper's labelled readout items. The scores for the "Ayonrinde & Lindsey" lenses are from koayon/jpp-lenses. We scored the "Neuronpedia" lenses with the paper's evaluation, on its held-out items.

*The poetry task is dropped for Gemma 4 31B, where only 1 of its 98 items passes the correctness filter.

File format

Each <model>_jpp_lens.pt is a torch.save dictionary, loadable with torch.load(path, weights_only=True):

Key	Value
`J`	`{layer: fp16 Tensor[d_model, d_model]}`, the map from that layer's residual stream to the final block's
`source_layers`	the layers in `J`: every block except the final one
`d_model`	the residual width
`n_prompts`	the number of WikiText-103 prompts in the fit
`provenance`	the source file, the method, the fitting configuration, and the activation that the maps read (`capture_point`, `stream_reduce`)

The readout at layer ℓ for a residual h is logits = unembed(h @ J[ℓ].T), where unembed is the model's final norm followed by its LM head. For DeepSeek-V4-Flash, h is the mean of the four residual streams.

The config.yaml beside each file records the source, the fitting configuration, and the command that wrote the file.

Licence

The J++ Lens files (<model>/jpp/) are Apache-2.0, as the source lenses. Each lens is derived from its base model. The method builds on the Jacobian Lens of Gurnee et al. (2026) and the R-Lens of Camila Blank and Agam Bhatia.

Citation