Cloud and cloud-shadow segmentation for CHRIS/PROBA-1 imagery, with three
classes: clear, cloud and shadow. It returns a label map for the whole
scene.
A CHRIS file is not four bands. It holds dozens of narrow bands (36 in the mode 3
example), in an order that changes with the acquisition mode, and their centres
rarely land on the wide Sentinel-2 red, green and blue. The model always takes
the same four inputs, close to Sentinel-2 B4, B3, B2 and B8, so for every mode the
loader first groups the CHRIS bands into those four channels, weighting them by
how well their spectral response matches the Sentinel-2 band. The weights are in bands.json, one entry per mode. Nothing
else has to be done by hand: predict_chris reads the raw cube, builds the four
channels and segments the scene.
pip install mlstac pytorch-lightning segmentation-models-pytorch timm rasterio
import mlstac
model = mlstac.load("https://huggingface.co/isp-uv-es/CHRIS-PROBA1/resolve/main/mlm.json")
local = model.download("CHRIS-PROBA1")
net = local.compiled_model(device="cuda")
mask = local.module.predict_chris("scene_TOA.tif", model=net, mode_n=3, source="toa")
# 0 = clear, 1 = cloud, 2 = shadow, 255 = nodata
On 55 hand-painted CHRIS scenes, split by flight and never seen in training,
each scene going through the full inference path in load.py:
| Metric | Value | Aggregation |
|---|---|---|
| mIoU | 0.708 | mean per scene |
| IoU clear / cloud / shadow | 0.898 / 0.831 / 0.652 | pixels pooled |
| Macro F1 | 0.881 | pixels pooled |
| Cohen's kappa | 0.854 | pixels pooled |
| Accuracy | 0.921 | pixels pooled |
The first row averages scenes; the rest pool every valid pixel. They are not interchangeable: the pooled mIoU is 0.793, the per-scene mean 0.708.
Per mode, mean mIoU and number of scenes: mode 1 0.784 (12), mode 2 0.697 (5), mode 3 0.699 (12), mode 4 0.576 (5), mode 5 0.658 (3), mode 20 0.711 (18). Modes 2, 4 and 5 have five scenes or fewer, so those numbers are indicative only.
Three U-Nets, averaged:
| Checkpoint | Encoder |
|---|---|
checkpoints/convnext_a.ckpt |
ConvNeXtV2-nano |
checkpoints/convnext_b.ckpt |
ConvNeXtV2-nano |
checkpoints/regnety.ckpt |
RegNetY-004 |
The two ConvNeXt share architecture and bands; they differ in the labels they learned from, so their errors do not line up.
Inference walks the scene in 640 px windows with 320 px overlap, snaps the last window to the edge so nothing is padded, normalises each window per band with a z-score, averages the softmax of the three checkpoints and blends the windows with a gradient mask. Around 0.3 s per 748 x 744 scene on an RTX A5500 with the weights loaded, 4 s on CPU.
Each mode has its own set of CHRIS bands and weights. Mode 3, for example, builds B4 from two adjacent bands, while mode 1 needs four bands for B4 and twelve for B8. To see what a mode uses:
local.module.band_table()["3"]
The file you pass can be in three radiometric states:
source |
What the file holds |
|---|---|
"toa" |
top-of-atmosphere reflectance, scaled by 10000 |
"rad" |
radiance as the sensor recorded it |
"dn" |
raw digital numbers, uncalibrated |
All three load, because the per-window z-score removes the global scale. Values
are divided by 10000 and clipped at 2.0 before that, so extreme radiance can
saturate. TOA and radiance are the accurate ones; DN is a fallback. Mode 20 is
passed as mode_n=20 and is normally delivered as radiance.
The input is a GeoTIFF with the CHRIS bands in their native order, counted from
Modes 1, 2, 3, 4, 5 and 20 are supported. Mode 1 is binned to 372 x 374 by the instrument, so its window shrinks to 352 px; the others are 744 px wide, with 748 rows in most acquisitions and up to 1024 in some mode 2 products.
With your own four-channel stack:
mask = local.module.predict_large(rgbn, model=net) # rgbn: (4, H, W) float
Per-pixel confidence:
probs = local.module.predict_chris("scene_TOA.tif", model=net, mode_n=3,
source="toa", return_probs=True)
Walkthrough with the six modes, a GPU against CPU timing and a batch example:
notebook.ipynb. The Colab badge at the top opens an
editable copy in Google Drive.
cloud.load.py mlstac loader
mlm.json model card, STAC/mlm format
bands.json CHRIS to RGBN table per acquisition mode
checkpoints/ the three networks
examples/mode_<n>/ one scene per mode (mode_20 for 20), mask and prediction
figures/ metrics, example, gallery
notebook.ipynb Colab demo
metrics.json per-class and per-mode numbers
MIT licence.
back_v1/convnextv2.ckpt 224 MBcheckpoints/convnext_a.ckpt 224 MBcheckpoints/convnext_b.ckpt 224 MBback_v1/regnety.ckpt 81 MBcheckpoints/regnety.ckpt 81 MB