BERT-Base (enlighten_compressor) — NeuralOps Compressed & LoRA-Healed Model Zoo

This repository provides a physically compressed and LoRA-healed checkpoint of google-bert/bert-base-uncased fine-tuned on GLUE SST-2, compressed using the enlighten_compressor (ENLightenCompressor) engine from the NeuralOps Library at Pareto-selected rank k=128 (2.39x compression ratio, -58.2% parameter reduction).

Compression & Architecture Profile

ENLightenCompressor: ENLightenCompressor neural operator. ENLighten: Low-Rank SVD + 2:4 Structured Sparsity (Photonic-Aligned). Registered as 'enlighten' and 'enlighten_compressor'. Production hardened for multi-architecture compatibility (DistilBERT, RoBERTa, GPT-2, Qwen3.5) with vectorization, error validation, and safe fallback.

Property	Specification
Registry Key (`COMPRESSOR_REGISTRY`)	`enlighten_compressor`
NeuralOps Class & Family	`ENLightenCompressor` (`low_rank`)
Source Implementation	`neuralops/compression/low_rank/enlighten_compressor.py`
Base Architecture	`google-bert/bert-base-uncased` (`109,483,778` parameters, 12 layers, $D=768$)
Showcase Rank (`k`)	`128` (`LoRA Recovery Applied: True`)
Compressed Parameter Count	`45,782,786` (`-58.18%` reduction, `2.391x` physical compression ratio)
Teacher Baseline SST-2 Acc	`91.63%`
Showcase Final SST-2 Acc	`71.90%` (Zero-Shot: `52.41%`, Parity Delta: `-19.73%`)

Multi-Rank Pareto Evaluation (`k=256, 128, 64, 32`)

Evaluated physically on NVIDIA GeForce RTX 3090 (24GB) on the GLUE SST-2 validation split (872 samples). Whenever zero-shot compression drops accuracy by > 2.0% alongside > 1.10x reduction, NeuralOps applies targeted r=8 LoRA healing (50 steps, lr=2e-4, 45s budget):

Rank (`k`)	Physical Ratio	Compressed Params (Reduction)	Zero-Shot Acc	Healed Acc	Parity vs Teacher (`91.63%`)
`256`	`1.63x`	`67,016,450` (`-38.8%`)	`59.52%`	`56.08%` (LoRA)	`-35.55%`
`128` ⭐	`2.39x`	`45,782,786` (`-58.2%`)	`52.41%`	`71.90%` (LoRA)	`-19.73%`
`64`	`3.11x`	`35,165,954` (`-67.9%`)	`51.38%`	`50.92%` (LoRA)	`-40.71%`
`32`	`3.67x`	`29,857,538` (`-72.7%`)	`50.11%`	`69.15%` (LoRA)	`-22.48%`

(⭐ indicates the serialized checkpoint packaged in this Hugging Face repository.)

Packaged Artifacts

pytorch_model.bin: Full serialized compressed BERT state dictionary (with low-rank factors / quantization buffers / LoRA healing adapters baked in).
config.json: Model configuration tagged with compression_method: "enlighten_compressor" and compression_rank: 128.
compression_summary.json: Complete machine-readable telemetry across all evaluated ranks [256, 128, 64, 32].
Tokenizer Assets: Standard bert-base-uncased vocabulary (vocab.txt, tokenizer.json, tokenizer_config.json).

Quick Start (Reconstruction & Inference)

import torch
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer, BertForSequenceClassification
from neuralops.core.compression_pipeline import compress_model

repo_id = "tahamajs/bert-comp-enlighten-compressor-sst2"
tokenizer = AutoTokenizer.from_pretrained(repo_id)

# Reconstruct compressed NeuralOps topology and load serialized compressed + healed weights
model = BertForSequenceClassification.from_pretrained("google-bert/bert-base-uncased", num_labels=2)
model = compress_model(model, {"compressors": [{"method": "enlighten_compressor", "rank": 128}]})

ckpt_path = hf_hub_download(repo_id=repo_id, filename="pytorch_model.bin")
state_dict = torch.load(ckpt_path, map_location="cpu")
model.load_state_dict(state_dict, strict=False)
model.eval()

inputs = tokenizer("NeuralOps achieves strong compression without sacrificing downstream accuracy.", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
    pred = logits.argmax(dim=-1).item()
print("Predicted SST-2 Sentiment (0=Negative, 1=Positive):", pred)

Citation

@software{neuralops2026compressorzoo,
  title   = {NeuralOps: Multi-Rank Compressed BERT-SST2 Zoo with Pareto LoRA Recovery},
  author  = {Majlesi, Taha},
  year    = {2026},
  url     = {https://github.com/tahamajs/NeuralOps}
}