Loading…
Loading…
google-bert/bert-base-uncasedThis repository provides a physically compressed and LoRA-healed checkpoint of google-bert/bert-base-uncased fine-tuned on GLUE SST-2, compressed using the rpca_compressor (RPCACompressor) engine from the NeuralOps Library at Pareto-selected rank k=128 (2.39x compression ratio, -58.2% parameter reduction).
RPCACompressor: RPCACompressor neural operator. Robust PCA: decompose W = L + S, factorizing low-rank component L with structural SVDLinear. Registered in COMPRESSOR_REGISTRY as 'rpca', 'rpca_factory', and 'rpca_compressor_factory'.
| Property | Specification |
|---|---|
Registry Key (COMPRESSOR_REGISTRY) |
rpca_compressor |
| NeuralOps Class & Family | RPCACompressor (low_rank) |
| Source Implementation | neuralops/compression/low_rank/rpca_compressor.py |
| Base Architecture | google-bert/bert-base-uncased (109,483,778 parameters, 12 layers, $D=768$) |
Showcase Rank (k) |
128 (LoRA Recovery Applied: True) |
| Compressed Parameter Count | 45,792,002 (-58.17% reduction, 2.391x physical compression ratio) |
| Teacher Baseline SST-2 Acc | 91.63% |
| Showcase Final SST-2 Acc | 72.71% (Zero-Shot: 52.87%, Parity Delta: -18.92%) |
k=256, 128, 64, 32)Evaluated physically on NVIDIA GeForce RTX 3090 (24GB) on the GLUE SST-2 validation split (872 samples). Whenever zero-shot compression drops accuracy by > 2.0% alongside > 1.10x reduction, NeuralOps applies targeted r=8 LoRA healing (50 steps, lr=2e-4, 45s budget):
Rank (k) |
Physical Ratio | Compressed Params (Reduction) | Zero-Shot Acc | Healed Acc | Parity vs Teacher (91.63%) |
|---|---|---|---|---|---|
256 |
1.63x |
67,034,882 (-38.8%) |
59.75% |
50.92% (LoRA) |
-40.71% |
128 ⭐ |
2.39x |
45,792,002 (-58.2%) |
52.87% |
72.71% (LoRA) |
-18.92% |
64 |
3.11x |
35,170,562 (-67.9%) |
51.38% |
49.08% (LoRA) |
-42.55% |
32 |
3.67x |
29,859,842 (-72.7%) |
49.43% |
66.74% (LoRA) |
-24.89% |
(⭐ indicates the serialized checkpoint packaged in this Hugging Face repository.)
pytorch_model.bin: Full serialized compressed BERT state dictionary (with low-rank factors / quantization buffers / LoRA healing adapters baked in).config.json: Model configuration tagged with compression_method: "rpca_compressor" and compression_rank: 128.compression_summary.json: Complete machine-readable telemetry across all evaluated ranks [256, 128, 64, 32].bert-base-uncased vocabulary (vocab.txt, tokenizer.json, tokenizer_config.json).import torch
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer, BertForSequenceClassification
from neuralops.core.compression_pipeline import compress_model
repo_id = "tahamajs/bert-comp-rpca-compressor-sst2"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
# Reconstruct compressed NeuralOps topology and load serialized compressed + healed weights
model = BertForSequenceClassification.from_pretrained("google-bert/bert-base-uncased", num_labels=2)
model = compress_model(model, {"compressors": [{"method": "rpca_compressor", "rank": 128}]})
ckpt_path = hf_hub_download(repo_id=repo_id, filename="pytorch_model.bin")
state_dict = torch.load(ckpt_path, map_location="cpu")
model.load_state_dict(state_dict, strict=False)
model.eval()
inputs = tokenizer("NeuralOps achieves strong compression without sacrificing downstream accuracy.", return_tensors="pt")
with torch.no_grad():
logits = model(**inputs).logits
pred = logits.argmax(dim=-1).item()
print("Predicted SST-2 Sentiment (0=Negative, 1=Positive):", pred)
@software{neuralops2026compressorzoo,
title = {NeuralOps: Multi-Rank Compressed BERT-SST2 Zoo with Pareto LoRA Recovery},
author = {Majlesi, Taha},
year = {2026},
url = {https://github.com/tahamajs/NeuralOps}
}