SigLIP — colorectal tissue classification (NCT-CRC-HE)
Nine-class tissue classifier for 224 × 224 px H&E patches of colorectal cancer tissue. Trained with one fixed recipe that we apply unchanged to every dataset, without dataset-specific tuning.
95.74 % accuracy on patients the model never saw — above the dataset authors' own model (94.3 %) and the classic CNN baselines, within 0.2 points of self-supervised ViTs, trained on about one third of the available images.
| Metric on CRC-VAL-HE-7K (7,180 patches) | Value |
|---|---|
| Accuracy | 95.74 % |
| Balanced accuracy | 94.58 % |
| Macro AUC (one-vs-rest) | 0.982 |
The dataset
NCT-CRC-HE is one of the most used public benchmarks in computational pathology: nine tissue classes — adipose, background, debris, lymphocytes, mucus, smooth muscle, normal mucosa, stroma, tumour epithelium.
| Split | Patches | Role |
|---|---|---|
| NCT-CRC-HE-100K | 100,000 (86 whole-slide images) | training |
| CRC-VAL-HE-7K | 7,180 (separate patients) | external test |
The problems with this dataset
1. The random-split trap
The training set has no slide or patient identifiers, so a random train/test split puts patches from the same slide on both sides. On a random 10 % of NCT-CRC-HE-100K this model scores 99.7 %; on the external test set it scores 95.7 %. Only the external number is a fair measure — it is the one reported everywhere on this page.
2. A colour shortcut
Ignatov & Malivenko (2024) showed that three numbers per image — mean red, green and blue — already give 53.8 % accuracy, and a colour histogram 82.2 %, without looking at a single cell.
3. JPEG artifacts and corrupted patches
The same study found class-dependent JPEG compression and patches corrupted by wrong dynamic-range handling after stain normalisation — signals a model can learn instead of tissue morphology.
What others reported on CRC-VAL-HE-7K
Same protocol for every method: trained on NCT-CRC-HE-100K, tested on CRC-VAL-HE-7K.
| Method | Accuracy |
|---|---|
| DINO · self-supervised ViT | 95.9 % |
| iBOT · self-supervised ViT-L | 95.8 % |
| SigLIP (this model) | 95.74 % |
| VGG16 | 95.3 % |
| ResNet-50 | 94.8 % |
| Inception-v3 | 94.8 % |
| VGG19 — Kather et al. 2019 (dataset paper) | 94.3 % |
| DenseNet | 92.9 % |
Methods designed specifically around this dataset's colour and artifact issues score higher (up to 98.33 % for an ensemble of two EfficientNet-B0 models). All published values as compiled in Ignatov & Malivenko 2024, Table 2.
Results in detail
| Class | Recall |
|---|---|
| Adipose | 98.1 % |
| Background | 100.0 % |
| Debris | 97.9 % |
| Lymphocytes | 100.0 % |
| Mucus | 96.7 % |
| Smooth muscle | 83.6 % |
| Normal mucosa | 98.5 % |
| Stroma | 80.3 % |
| Tumour epithelium | 96.1 % |
Stroma and smooth muscle — two fibrous tissues that look alike at this magnification — are the remaining hard pair, for this model and for every published model.
Training and evaluation
| Training data | at most 4,000 patches per class from NCT-CRC-HE-100K (≈ 36,000 of 100,000) |
| Training | 15 epochs, learning rate 5e-5, batch 16, minority classes oversampled |
| Model selection | best epoch on a held-out 10 % of NCT-CRC-HE-100K |
| Test | CRC-VAL-HE-7K, evaluated once, no tuning on the test set |
| Hardware | one NVIDIA T4 |
Usage
from PIL import Image
import torch
from torchvision.transforms import Compose, Normalize, Resize, ToTensor
from transformers import AutoModelForImageClassification
model = AutoModelForImageClassification.from_pretrained("ghostsas001/nct-crc-siglip").eval()
tf = Compose([Resize((224, 224)), ToTensor(), Normalize([0.5] * 3, [0.5] * 3)])
x = tf(Image.open("patch.jpg").convert("RGB")).unsqueeze(0)
probs = torch.softmax(model(pixel_values=x).logits, 1)[0]
print(model.config.id2label[int(probs.argmax())])
Labels: ADI, BACK, DEB, LYM, MUC, MUS, NORM, STR, TUM.
Limitations
Research model, not a medical device. Trained on Macenko-normalised patches from one source; other scanners, stains or magnifications may reduce accuracy. One training run, no confidence intervals.
References
- Kather J.N. et al. Predicting survival from colorectal cancer histology slides using deep learning. PLoS Medicine 16(1): e1002730, 2019. doi:10.1371/journal.pmed.1002730
- Dataset: NCT-CRC-HE-100K and CRC-VAL-HE-7K, Zenodo 1214456
- Ignatov A., Malivenko G. NCT-CRC-HE: Not All Histopathological Datasets Are Equally Useful. 2024. arXiv:2409.11546