IDiomSAE-300M-L18-k32
IDiomSAE is a Top-K sparse autoencoder trained on the residual stream of layer-18 of IDiom-300M. It supports IDR feature analysis and generation steering.
Preprint | IDiom GitHub | IDiom Models | IDiom-DB dataset
Model details
| Property | Value |
|---|---|
| Base model | IDiom-300M |
| Layer | 18 |
| Input width | 1,024 |
| Latent features | 16,384 |
| Sparsity | Top-K, k = 32 per residue |
| Training input | IDR residues in unprompted FIM format |
See the IDiom-300M model card for more information.
Usage
Please see the IDiom GitHub repository for usage information.
License
Model weights and code: MIT.