OpenMidnight

Overview of the OpenMidnight pathology foundation model

State-of-the-art pathology foundation model trained on 12K slides

Developed by Sophont

Blog Post | GitHub | Demo

---

What is OpenMidnight?

OpenMidnight is our open replication of Kaiko.AI's Midnight, a 1.1 billion parameter Vision Transformer foundation model for computational pathology. OpenMidnight achieves state-of-the-art performance despite being trained on significantly less data than Kaiko.AI's Midnight or comparable models.

Key advantages:

• 🏆 State-of-the-art performance: Achieves 0.775 average score across 14 benchmarks
• ⚡ Efficient training: Trained in ~83 hours on 8× H100 GPUs for only $1,600 USD (estimated)
• 📊 Minimal data requirements: Uses only 12K slides from TCGA for training
• 🔓 Fully open source: Complete model weights, training code, and pipeline publicly available

OpenMidnight is aimed for computational pathology tasks including:

• Tumor detection and classification
• Histological grading
• Tissue segmentation
• Margin assessment
• Clinical outcome prediction

---

Model Description

• Developed by: Sophont
• Model type: Finetuned DINOv2 ViT-G for H&E pathology images
• Training data: TCGA, 12M H&E WSI
• Training repository: https://github.com/MedARC-AI/OpenMidnight/tree/main

Usage

Requirements

pip install torch torchvision huggingface_hub

Recommended: Run on GPU with mixed precision for optimal performance.

Quick Start: Loading the Model

import torch
from huggingface_hub import hf_hub_download

#Downloads to hf cache location
download_location = hf_hub_download(repo_id="SophontAI/OpenMidnight", filename="teacher_checkpoint_load.pt")
model = torch.hub.load('facebookresearch/dinov2', 'dinov2_vitg14_reg', weights = None)

#Load OpenMidnight weights
checkpoint = torch.load(download_location, map_location = "cpu")

#Required because dinov2 is baseline 392 and we are baseline 224 resolution
pos_embed = checkpoint["pos_embed"]
model.pos_embed = torch.nn.parameter.Parameter(pos_embed)
model.load_state_dict(checkpoint)
model.eval()

print(f"Model loaded with {sum(p.numel() for p in model.parameters()):,} parameters")

Extracting Embeddings from Tissue Patches

from PIL import Image
import torchvision.transforms as transforms

# Standard preprocessing for pathology images
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(
        mean=[0.485, 0.456, 0.406],  # ImageNet normalization
        std=[0.229, 0.224, 0.225]
    )
])

# Load and preprocess an H&E tissue patch
image = Image.open("path/to/tissue_patch.jpg")
input_tensor = transform(image).unsqueeze(0)  # Shape: [1, 3, 224, 224]

# Extract embeddings
with torch.no_grad():
    embeddings = model(input_tensor)  # Shape: [1, 1536]

print(f"Embedding shape: {embeddings.shape}")
print(f"Embedding norm: {embeddings.norm().item():.4f}")

---

Model Performance

OpenMidnight achieves competitive or superior performance compared to models trained on 8-30× more data:

Benchmark Comparison

Average performance of top pathology foundation models and baselines across computational pathology benchmarks

Detailed Benchmark Results

Performance comparison of OpenMidnight to existing pathology foundation models on eva+HEST benchmarks. Scores for existing models are taken from Midnight paper. We report balanced accuracy for the classification tasks, Dice score for semantic segmentation (CoNSeP and MoNuSAC), and the average Pearson correlation for the nine HEST regression tasks. Only performance with [CLS] token is reported. Best score per dataset is bolded.

---

Model Details

Architecture

Training Data

• Dataset: TCGA (The Cancer Genome Atlas)
• Slides: 12k FFPE H&E-stained whole slide images
• Cancer Types: 32 different cancer types
• Total Patches: 96 million
• Unique Patches: 29 million
• Stain Type: Hematoxylin and Eosin (H&E)
• Preprocessing: Non-informative patch filtering

Training Configuration

• Hardware: 8× NVIDIA H100 GPUs (80GB each)
• Batch Size: 48 per GPU (384 global batch size)
• Training Steps: 250,000
• Optimizer: AdamW
• Learning Rate: 2.0e-4
• Regularization: KDE regularizer for training stability
• Augmentation: Hematoxylin-Eosin-DAB colorspace transformations
• Training Time: ~83 hours wall-clock time (667 GPU-hrs)
• Training Cost: ~$1,600 USD (at $2.50/H100/hour)

---

Blog Post

For an in-depth discussion of OpenMidnight, read the full blog post.

---

Contact

For questions, feedback, or collaboration opportunities:

• 📧 Email: contact@sophont.med
• 🌐 Website: sophont.med
• 🐦 Twitter/X: @SophontAI
• 💬 GitHub Issues: github.com/MedARC-AI/OpenMidnight/issues

We welcome:

• Bug reports and feature requests
• Contributions to the training code
• Benchmark results on new datasets
• Applications of OpenMidnight to novel tasks

---

Acknowledgments

We thank Mikhail Karasikov for answering questions about Midnight. We thank Nicolas Känzig for answering questions about eva. We thank the members of MedARC and the broader research community for their feedback and support. We are very grateful to FAL AI for granting compute to support this open-source research.

---

Citation

If you use OpenMidnight in your research, please cite:

@article{kaplan2025openmidnight,
            author = {Kaplan, Daniel and Grandhi, Ratna Sagari and Lane, Connor and Warner, Benjamin and Abraham, Tanishq Mathew and Scotti, Paul S.},
            title = {How to Train a State-of-the-Art Pathology Foundation Model with \$1.6k},
            year = {2025},
            url = {https://sophont.med/blog/openmidnight},
          }

---

License

This model is released under the Apache 2.0 License.

---

Terms of Use

Research Use: This model is primarily intended for research purposes in computational pathology, medical imaging, and related fields.

Clinical Use: This model is not intended for use in medical diagnosis, treatment, or prevention of disease of real patients. It should not be used as a substitute for professional medical advice.

Responsible Use: Users should:

• Validate model performance on their specific use cases
• Be aware of potential biases in the training data (TCGA)
• Consider demographic and geographic limitations
• Respect privacy rights and comply with applicable data protection laws
• Follow applicable regulations and ethical guidelines

---