ESM-2 35M Sparse 50% — PharmaCore

A 50% magnitude-pruned version of facebook/esm2_t12_35M_UR50D optimized for efficient drug discovery inference on Apple Silicon.

Why This Model?

Use Case

Primary protein encoder in the PharmaCore drug discovery pipeline:

• Higher-capacity protein embeddings for drug-target compatibility
• De novo drug discovery and drug repurposing workflows
• Full audit trail support for regulatory transparency
• Runs entirely on consumer Apple Silicon hardware (M1/M2/M3/M4)

Usage

from transformers import AutoModel, AutoTokenizer
import torch

model = AutoModel.from_pretrained("stephenjun8192/esm2-35m-sparse50")
tokenizer = AutoTokenizer.from_pretrained("facebook/esm2_t12_35M_UR50D")

# Encode a protein target (e.g., EGFR kinase domain)
sequence = "MRPSGTAGAALLALLAALCPASRALEEKKVCQGTSNKLTQLGTFEDHFLSLQRMFNNCEVVL"
inputs = tokenizer(sequence, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)
    embedding = outputs.last_hidden_state.mean(dim=1)  # [1, 480]

print(f"Embedding shape: {embedding.shape}")

Sparsification Method

• Technique: Global magnitude pruning (unstructured)
• Sparsity: 50% of all weight parameters set to zero
• Layers pruned: All linear layers (attention Q/K/V/O, FFN)
• Validation: Cosine similarity of embeddings vs dense model ≥ 0.973

Benchmarks (Apple M4 Mac mini, 16GB)

Part of PharmaCore

PharmaCore — the first AI drug discovery platform that runs entirely on a MacBook. No cloud GPUs, no API keys, no data leaves your machine.

Citation

@software{pharmacore2026,
  title={PharmaCore: Apple Silicon-Native AI Drug Discovery},
  author={Stephen Wu},
  year={2026},
  url={https://github.com/reacherwu/PharmaCore}
}