Chunker + RC aug. (control) (chunker-rcaug)
The non-equivariant control: same architecture as MirrorDNA-duplex, not symmetrized, trained with reverse-complement augmentation.
MirrorDNA is a lightweight DNA language model with a learned tokenizer that is exactly equivariant to the reverse complement: a symmetric boundary router cuts a sequence and its reverse complement into mirror-image chunks for every value of the parameters, so both strands of the same DNA get the same tokens. This model is NOT strand-equivariant; it is the control trained with reverse-complement augmentation.
- Paper: Learning Strand-Equivariant Tokenizations of DNA with MirrorDNA
- Code: https://github.com/NIRJHOR-DATTA/MirrorDNA
- Parameters: 2.52M
- Pretraining: masked language modelling on hg38 (Basenji/Enformer training intervals), 1,024-bp windows, 2^20 bases per step for 10,000 steps (10.5B bases), 15% of bases masked.
Usage
import json, sys, torch
from huggingface_hub import snapshot_download
path = snapshot_download("nirjhor123/chunker-rcaug")
sys.path.insert(0, path)
import rcchunk as rk # the model code is stored with the weights
cfg = json.load(open(f"{path}/config.json"))
model = rk.build(cfg["variant"], cfg["model"])
from safetensors.torch import load_file
model.load_state_dict(load_file(f"{path}/model.safetensors"))
model.eval()
x = torch.from_numpy(rk.encode("ACGT" * 256)).long()[None] # [batch, length]; A,C,G,T,N = 0..4
out = model(x, decode=False)
out["cut"] # chunk boundaries (gap j = between base j and j+1)
model.pooled_features(out) # strand-invariant sequence features
model.pooled_features(out, oriented=True) # orientation-aware features, for strand-dependent labels
Fine-tuning scripts, the benchmarks and every analysis of the paper are in the GitHub repository.
Intended use and limitations
A research model for DNA sequence classification and regression, mainly with fine-tuning. It was pretrained on the human genome only, with 1-kb windows, and is small; larger models are more accurate on several benchmarks (see the paper). Strand-invariant features suit labels that do not depend on the strand; for strand-dependent labels (promoters, splice sites, gene orientation) use the orientation-aware features.
Citation
will be uploaded soon