DELIGHT-FM Immune
Pretrained foundation model for new downstream cell classification/regression tasks and embeddings. Preserves the masked-gene and auxiliary pretraining heads.
Repository: chloehlee/delight-fm-immune · License: Apache 2.0 · Selected foundation epoch: 3.5.
Start here
For log2(PCW) prediction or age perturbations, use chloehlee/delight-fm-immune-age. The foundation model's auxiliary PCW score is a learned plateau scale, not calibrated log2(PCW).
Notebooks use explicit published repository IDs and download matching code/resources; they do not depend on your working directory. Set your own data paths and annotation names. Human tissue-matched single-cell RNA expression is required; original study cells are not distributed.
Load this model
Use Python 3.10, the tested PyTorch/CUDA 11.8 stack and compiled Mamba extensions
described in the notebooks' Environment section and requirements.txt.
from transformers import AutoModel
model = AutoModel.from_pretrained(
"chloehlee/delight-fm-immune", trust_remote_code=True)
For preprocessing and the complete inference workflow, follow the linked notebook.
Review the custom Python before enabling trust_remote_code.
Inputs and saved outputs
Inputs comprise gene token IDs, float32 per-gene z-scored log-normalized expression
(clipped to ±5), and aligned HVG masks. Use the matching reference gene statistics
and tokenization files in resources/; resources/preprocessor_config.json records the
historical initial-ranking backend.
This foundation checkpoint does not contain the final fitted log2-PCW regression head. Fine-tuning creates a new task head while transferring every pretrained backbone tensor.
DELIGHT-FM model family
| Foundation | Task-specific companion |
|---|---|
| DELIGHT-FM | Saved broad-cell head included |
| DELIGHT-FM Epi-ileum | Age companion |
| DELIGHT-FM Epi-colon | Age companion |
| DELIGHT-FM Stromal | Age companion |
| DELIGHT-FM Immune | Age companion |
Selection and limits
Training subset: 175031 cells. Validation/test subset: 36586 cells. Reference gene features: 18253; context length: 2048. Pretraining combines masked gene modeling with auxiliary cell-cluster/broad-cell and learned age-scale supervision and adversarial batch/domain heads; all saved heads are preserved in the foundation export. Age companions use a separate fitted regression head.
Selection maximized the mean of cluster macro F1, location macro F1 and age Pearson,
excluding explicit single-class tasks. See metadata/selection.json for component values.
These are validation-selected results; the original test subset participated in model
selection. New downstream tasks should use donor-disjoint train/validation/test partitions
and evaluate generalization independently.
Perturbations measure model response, not experimentally established causal effects. The included notebooks evaluate saved classification heads or fit and save a new classifier. Gene and reconstructed-regulon age effects are documented in the matching age-model repository.
Reproducibility and validation
Weights are unchanged from the final source checkpoint.
Classification notebooks use this saved foundation model. Fine-tuning transfers its backbone to a new task head and saves the fitted classifier for strict reloading.
Model source metadata and selection records are in metadata/.
Credits and license
Published by chloehlee as part of DELIGHT-FM.
Release code and model weights: Apache 2.0. Reference assets and dependencies retain their original terms; see third-party attribution.
Full workflow and supporting data
Training, downstream evaluation and perturbation workflows: ChloeHJ/delight_fm_v1. Supporting training/expression inputs are staged for chloehlee/delight_fm_v1-data; this model package contains its own inference code, weights, preprocessing resources and tutorials.
Task-specific notebooks
- Saved-head cell-type and cluster classification.
- Fine-tune this foundation model for classification.
This foundation package provides saved broad-cell and cluster heads for immune cells, plus a notebook for fitting new cell-type or cluster classifiers. Location classification is available by fitting a new task head.
Bundled resources/regulons.tsv is the matching immune SCENIC+ reconstructed regulon table retained as a compartment reference. Gene and SCENIC+ age-perturbation workflows are hosted exclusively in chloehlee/delight-fm-immune-age, using that repository's fitted age head and fine-tuned backbone. The notebooks here use this foundation model for classification and task fine-tuning.
Age prediction and age perturbation tutorials are available exclusively in the matching age-model repository. Every notebook here loads this foundation model; new downstream classifiers are fitted from its backbone.