🚀 COPM: Converting clinical signals into outcome intelligence for DHTI.
dim256 CDS embedding model for quick prototyping and testing. Trained using contrastive learning with the Genomic Tokenizer on human protein-coding DNA sequences.
Note on Model Size: Due to the biologically-compressed nature of the Genomic Tokenizer, which operates at the codon level (26 tokens representing 20 amino acids plus special tokens), smaller embedding dimensions may actually outperform larger ones. The tokenizer's compression reduces the representation space complexity, enabling more efficient learning even with fewer parameters.
This model is part of a family of CDS embedding models:
| Variant | Parameters | Layers | Hidden Size | Embedding Dim | Use Case |
|---|---|---|---|---|---|
| dim128 | ~1M | 2 | 256 | 128 | ⭐ Preferred - efficient with tokenizer compression |
| dim256 | ~2M | 2 | 256 | 256 | Research |
| dim512 | ~4M | 4 | 512 | 512 | Research |
This model uses the Genomic Tokenizer which tokenizes DNA sequences at the codon level:
[CLS], [SEP], [BOS], [MASK], [PAD], [UNK]Each token represents a codon (3 nucleotides) that encodes a specific amino acid, enabling the model to learn at the protein-coding level rather than individual nucleotides.
Best for development, debugging, and small-scale experiments: ✅ Recommended:
pip install torch genomic-tokenizer pydantic
import torch
import json
from pathlib import Path
from huggingface_hub import hf_hub_download
from genomic_tokenizer import GenomicTokenizer
# You need the model classes (install gtembed package or copy model.py)
# from src.gtembed.model import CDSEmbeddingModel
# from src.gtembed.config import ModelConfig
# Download model files
# Replace with your variant: dim256, dim512, or dim768
repo_id = "beapen/genomic-cds-embedding-dim256-v1" # e.g., beapen/genomic-cds-embedding-dim256-v1
config_path = hf_hub_download(repo_id=repo_id, filename="config.json", repo_type="model")
model_path = hf_hub_download(repo_id=repo_id, filename="model.pt", repo_type="model")
# Load config
with open(config_path) as f:
config_dict = json.load(f)
# Initialize model (requires gtembed package)
# config = ModelConfig(**config_dict)
# model = CDSEmbeddingModel.from_config(config)
# model.load_state_dict(torch.load(model_path, map_location="cpu"))
# model.eval()
# Load tokenizer
tokenizer = GenomicTokenizer(model_max_length=2048)
import torch
# Sample CDS sequence (protein-coding DNA)
cds_sequence = "ATGGCCGCCTACAAGCAGCAGTCTGGGGAAGACGGGCTACCTTGCCCCTGA"
# Tokenize
tokens = tokenizer(cds_sequence)
input_ids = torch.tensor([tokens['input_ids']])
# Generate embedding
with torch.no_grad():
embedding = model(input_ids)
print(f"Embedding shape: {embedding.shape}") # [1, 256]
print(f"Embedding norm: {embedding.norm().item():.4f}") # Should be ~1.0 (L2 normalized)
sequences = [
"ATGGCCGCCTACAAGCAGCAGTCTGGGGAAGACGGGCTACCTTGCCCCTGA",
"ATGGCCGCCTACAAGCAGCAGTCTGGGGAAGACGGGCTACCTTGCCCCTGA",
# ... more sequences
]
# Tokenize all
tokenized = tokenizer(sequences, padding=True, return_tensors="pt")
input_ids = tokenized['input_ids']
attention_mask = tokenized['attention_mask']
# Generate embeddings
with torch.no_grad():
embeddings = model(input_ids, attention_mask=attention_mask)
print(f"Batch embeddings shape: {embeddings.shape}") # [batch_size, 256]
This model is intended for research purposes in genomics and bioinformatics. Users should:
If you use this model in your research, please cite the following:
Eapen, B. (2026). GTEmbed: Genomic Tokenizer Embedding Pipeline. Hugging Face. https://huggingface.co/beapen/genomic-cds-embedding-dim256-v1
Eapen, B. (2026). Genomic Tokenizer: Codon-level Tokenization for CDS Sequences. GitHub. https://github.com/dermatologist/genomic-tokenizer
This model was trained using CDS sequences extracted from two publicly available datasets:
1. GENCODE v50 Human Gene Annotation
GENCODE provides comprehensive reference gene annotation for the human genome. For GENCODE v50 (released in 2023), please cite:
Frankish, A., Diekhans, M., Ferreira, A.M., Johnson, R., Jungreis, I., Loveland, J., Mudge, J.M., Sisu, C., Wright, J., Armstrong, J., Barnes, I., Berry, K., Bignell, A., Carbonell-Sala, S., Cerdán-Vélez, D., Cheng, Y., Davidson, C., Dyer, S.C., Eyrignoux, C., Fatima, R., Fernandez-Banet, J., Gall, A., Garcia Girón, C., Gonzalez, J.M., Greller, R., Hardy, M., Hunt, T., Ishiguro, H., James, B., Jayakumar, A., Juettemann, T., Kay, M., Kulesha, E., Mangan, R.M., Martin, F.J., Mohanan, S., Muir, P., Murphy, S., Nag, A., Overduin, B., Parker, A., Perry, M., Petty, T.L.P., Pujara, P., Raj, S., Sanchez-Castillo, D., Schmitt, B., Schuilenburg, H., Sealy, I., Shafer, R., Sharma, L., Slater, T., Smedley, D., Sobala, A., Spudich, A., Steed, E., Tang, Y., Thakuria, J., Tress, M.L., Vazquez, J., Wilder, S., Zadissa, A., Zerbino, D.R., Cunningham, F., Lander, A., Flicek, P., and Harrow, J. (2023). GENCODE: reference annotation for the human and mouse genomes in 2023. Nucleic Acids Research, 51(D1), D942-D949. https://doi.org/10.1093/nar/gkac1071
Alternative citation for latest GENCODE release (2025):
Mudge, J.M., Carbonell-Sala, S., Diekhans, M., Martinez, J.G., Hunt, T., Jungreis, I., Loveland, J.E., Arnan, C., Barnes, I., Bennett, R., Berry, K., Bignell, A., Cerdán-Vélez, D., Cochran, K., Cortés, L.T., Davidson, C., Donaldson, S., Dursun, C., Fatima, R., Hardy, M., Hebbar, P., Hollis, Z., James, B., Jiang, Y., Johnson, R., Kaur, G., Kay, M., Mangan, R.M., Maquedano, M.G., Gómez, L.M., Mathlouthi, N., Merritt, R., Ni, P., Palumbo, E., Perteghella, T., Pozo, F., Raj, S., Sisu, C., Steed, E., Sumathipala, D., Suner, M.M., Uszczynska-Ratajczak, B., Wass, E., Yang, Y., Zhang, D., Finn, R.D., Gerstein, M., Guigó, R., Hubbard, T.J.P., Kellis, M., Kundaje, A., Paten, B., Tress, M.L., Birney, E., Martin, F.J., and Frankish, A. (2025). GENCODE 2025: reference gene annotation for human and mouse. Nucleic Acids Research, 53(D1), D966-D975. https://doi.org/10.1093/nar/gkae1078
Data source:
gencode.v50.annotation.gtf.gz2. GRCh38.p14 Human Reference Genome Assembly
The Genome Reference Consortium (GRC) maintains the human reference genome assembly. For GRCh38.p14 (accession: GCF_000001405.40), cite as:
Genome Reference Consortium. (2022). Human genome assembly GRCh38.p14. RefSeq assembly accession: GCF_000001405.40. https://www.ncbi.nlm.nih.gov/assembly/GCF_000001405.40
For the original GRCh38 release (2013), you may also cite:
Church, D.M., Dunn, D.M., Hodgson, A., Loveland, J.E., Smith, A.N., Bevan, P., Cunningham, F., Earhart, S., Farrell, C.M., Gildert, J., Graf, S., Kay, M., Khurana, E., Li, Y., Thakuria, J., Trevanion, S.J., Veyrieras, J.-B., Vogel, J., White, S.J., Aken, B.L., Barrell, D., Brent, S., Chen, Z., Delcher, A., Frankish, A., Gietzen, D., Gilbert, J.G., Graham, U., Hammond, S., Haussler, D., Hermjakob, H., Hume, D.A., Kasprzyk, A., Kulesha, E., Lander, A., Long, J., Melsens, S., Meidl, P., Murphy, S., Nigro, J., Park, T., Rosenfeld, J.A., Schuster, S.C., Sealy, I., Souvorov, A., Souvorova, A., Steward, C., Taudien, S., Tress, M.L., Ureta-Vidal, A., Wallen, T., Weber, R., White, O., Woodward, J.R., Zadissa, A., Zdobnov, E.M., Zerbino, D.R., Blankenberg, D., Kent, W.J., Korf, I., Lander, E.S., Hubbard, T.J.P., Flicek, P., Harrow, J., Weber, R.J., Cooper, D.M., Eichler, E.E., Felsenfeld, G., Green, E.D., Grimwood, J., Guttmacher, C., Hardison, R.C., Istrail, S., Jaffe, D.B., Myers, R.M., Pennacchio, L.A., Ponting, C.P., Rubin, E.M., Ryder, O., Venter, J.C., Waterston, R.H., Wilson, R.K., and Collins, F.S. (2014). Grading the quality of genome assemblies. Nature Biotechnology, 32, 101-104. https://doi.org/10.1038/nbt.2783
Data source:
GRCh38.primary_assembly.genome.fa.gzBest practices for citing genome assemblies:
Apache 2.0
genomics, dna-sequences, embedding, transformer, contrastive-learning, protein-coding, cds, genomic-tokenizer, bioinformatics, healthcare, dim256-model