Final saved checkpoint for this run of the lexicon model accompanying Learned Sparse Lexical Representations for Large-scale Authorship Attribution.
Use the code in lingjzhu/neural_lexical.
The output has one vector per input token with 500 dimensions.
Token cluster assignments are learned during training and included in cluster_ids.pt.
Load them along with the weights; do not run k-means again for inference.
This checkpoint uses log1p(ReLU) token activations, without CLS/SEP tokens or embedding rescaling. Score with mean MaxSim over nonzero query tokens.
Clone the code repository and install its dependencies, then run from its root:
from src.models.ClusteredTransformer import ClusteredTransformer, clustered_maxsim
from src.models.SparseEncoder import SparseEncoder
import torch
module = ClusteredTransformer.load("UBC-SLIME/sparcol-lexicon-large-c500")
model = SparseEncoder(modules=[module]).to("cuda").eval()
features = model.tokenize(["A sample document.", "Another document."])
features = {k: v.to("cuda") for k, v in features.items()}
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
embeddings = model(features)["sparse_embeddings"]
scores = clustered_maxsim(embeddings, embeddings)
exp2/outputs_clustered_colbert_modernbert_large_500_relu_scale1_nocls/checkpoint-1580answerdotai/ModernBERT-largeWeights and cluster assignments are preserved from the source checkpoint. Packaging adds portable module configuration and removes training logs and training-data examples. Full HRS evaluation has not been rerun for this release; no new benchmark results are claimed.
model.safetensors 1.6 GBcluster_ids.pt 404.5 kBanswerdotai/ModernBERT-large