scalar_kl / seed 104

Grassmannian block-sparse featurizer trained from scratch for a fixed DINOv3/Imagenette patch classifier. Profile: full; completed epochs: 1. Scalar artifacts are tied signed width-one featurizers, not conventional ReLU SAEs.

Study-level interpretation: Bounded preregistered study; interpret completion and power status below. Exploratory: the pilot power target was not met. Mean paired local-minus-KL difference: 0.00254596 nats/image over 5 seed(s). KL training had lower mean output divergence. Seed-level 95% interval: [-0.00325909, 0.008351]. The interval includes zero; the directional comparison is inconclusive.

Measured held-out KL: 0.029281568 nats/image; accuracy: 98.9419%; normalized patch MSE: 0.79675543. Full metrics and per-image logits accompany this model.

Use

Install the project with uv sync --frozen, download this repository with hf download, then:

from PIL import Image
from e2e_bsf.inference import VisionFeaturizer
model = VisionFeaturizer("path/to/downloaded/model")
probabilities = model.predict([Image.open("image.jpg").convert("RGB")])

The artifact contains BSF weights, fixed normalization and the trained classifier. It excludes DINO weights and photos. Authorized DINO access and its separate license are required; the MIT license here does not relicense the backbone. Class order follows torchvision Imagenette.

Limits and provenance

Lower KL only measures preservation of this ten-class readout. It does not establish semantic concepts, full DINO fidelity, or inference acceleration. config.json binds revisions, architecture, objective, seed, preprocessing and training identities. manifest.json and duplicate_audit.json preserve image assignments and exclusion evidence. teacher.json records classifier selection. model.safetensors is for inference; the local training checkpoint additionally retains optimizer/RNG state.

Sources: BSF paper, end-to-end SAE paper, Goodfire code, preregistration.