🌍 General geospatial foundation model → 🎯 one controlled change → 🗺️ ISPRS Vaihingen land-cover segmentation
| 🧪 | |
|---|---|
| What is this? | KAO-TerraMind-A-Vaihingen is TerraMind-1.0-base fine-tuned on ISPRS Vaihingen with variant A: optimisation - stronger backbone adaptation. |
| Task | 6-class semantic segmentation · aerial IRRG, 9 cm |
| Test mIoU | 71.74 (+1.09 pp vs. the identically trained TerraMind-1.0-base, rank 2 / 7) |
| Overall accuracy | 85.72 % |
| Cost | 97.60 M parameters · 14.64 ms / 224 x 224 tile (T4, fp16, batch 1) |
| Purpose | 🔬 Public, fully-specified checkpoint for reproducing the CVPR 2027 submission |
🧾 Training code is intentionally not included while the paper is under review. This repository ships the inference-only helper, the exact architecture / recipe config, and the full benchmark results.
| Property | Value |
|---|---|
| 🏷️ Model | KAO-TerraMind-A-Vaihingen |
| 🧠 Backbone | TerraMind-1.0-base (terramind_v1_base), initialised from ibm-esa-geospatial/TerraMind-1.0-base |
| 🧩 Decoder | UperNet (TerraTorch EncoderDecoderFactory) |
| 🎞️ Input modality | RGB patch embedding · 3 x 224 x 224 · aerial IRRG (NIR, R, G), 8-bit, fed to the RGB modality in stored band order |
| 🎯 Output | per-pixel logits over 6 classes |
| 📐 Parameters | 97.60 M |
| 🧪 Variant | A - Optimisation: stronger backbone adaptation |
| 🗂️ Dataset | ISPRS Vaihingen - Aerial IRRG (NIR-R-G) orthophotos, 9 cm GSD, 33 tiles |
| 📦 Checkpoint file | terramind_v1_kaoA_best.pt |
| 🔧 Library | TerraTorch >=1.2.4 |
KAO-TerraMind is a family of three controlled, single-factor adaptations of TerraMind-1.0-base. Everything else (data, seed, epochs, batch size, sampler, schedule, metrics) is identical to the baselines, so every difference can be attributed to one change.
TerraMind-1.0-base (identical training recipe)
│
┌──────────────────┼──────────────────┐
▼ ▼ ▼
🔥 KAO-A 🎯 KAO-B 🧩 KAO-C
Optimisation Loss Decoder
backbone LR ↑ Dice weight ↑ wider UperNet
weight decay ↑ deeper taps
This repository = 🔥 KAO-A (Optimisation) - lets the pretrained TerraMind encoder move further away from its pre-training solution (3x larger backbone learning rate, 5x larger weight decay).
| Setting | TerraMind-1.0-base (reference) | KAO-A |
|---|---|---|
| Backbone LR multiplier | 0.1 | 0.3 |
| Weight decay | 0.01 | 0.05 |
| Hyper-parameter | Value |
|---|---|
| Optimiser | AdamW (backbone LR = base LR x multiplier, decoder LR = 1e-4) |
| Base learning rate / backbone multiplier | 1e-4 / 0.1 (variant A: 0.3) |
| Weight decay | 0.01 (variant A: 0.05) |
| Loss | class-weighted cross-entropy (sqrt-inverse-frequency weights) + Dice (weight 0.5; variant B: 1.0) |
| Epochs / early-stopping patience | 30 / 8 (checkpoint chosen on validation mIoU) |
| Batch size | 16 (same for every model) |
| Sampler | class-balanced, 6,000 windows per epoch |
| Window size / train stride | 224 x 224 / 112 px |
| Augmentation | random horizontal + vertical flips and 90° rotations |
| Precision / grad-clip | fp16 autocast / 1.0 |
| Normalisation | TerraMind RGB pre-training statistics (0-255 scale) |
| Seed | 42 |
| Hardware | 1 x NVIDIA Tesla T4 |
ISPRS Vaihingen - Aerial IRRG (NIR-R-G) orthophotos, 9 cm GSD, 33 tiles.
| ID | Class | Map colour | |
|---|---|---|---|
| 0 | 🛣️ | Impervious surfaces | #ffffff |
| 1 | 🏢 | Building | #0000ff |
| 2 | 🌱 | Low vegetation | #00ffff |
| 3 | 🌳 | Tree | #00ff00 |
| 4 | 🚗 | Car | #ffff00 |
| 5 | 🗑️ | Clutter | #ff0000 |
Mean metrics (mIoU / F1 / Prec. / Rec. / mAP) are averaged over the 5 foreground classes (Clutter excluded), the usual ISPRS convention. OA uses all pixels.
| Split | 224 x 224 windows |
|---|---|
| Train | 5,169 |
| Validation | 192 (up to 1,500 sampled per epoch for model selection) |
| Test | 1,994 |
📜 ISPRS Vaihingen is distributed for research use by ISPRS. Please check the dataset terms before using these weights beyond research.
All seven models are fine-tuned with the same recipe, seed and data, and evaluated on the same test windows. Best value per column in bold.
| Model | OA | mIoU | F1 | Prec. | Rec. | mAP | Params (M) | Latency (ms) |
|---|---|---|---|---|---|---|---|---|
| KAO-TerraMind-A ⬅ this repo | 85.72 | 71.74 | 83.17 | 83.95 | 82.55 | 89.59 | 97.60 | 14.64 |
| KAO-TerraMind-B | 85.09 | 70.79 | 82.53 | 82.88 | 82.26 | 89.99 | 97.60 | 14.64 |
| KAO-TerraMind-C | 85.04 | 70.96 | 82.67 | 83.10 | 82.37 | 89.75 | 119.97 | 17.63 |
| TerraMind-1.0-tiny | 82.32 | 65.37 | 78.40 | 79.41 | 77.69 | 85.71 | 12.87 | 13.58 |
| TerraMind-1.0-small | 82.51 | 66.02 | 78.99 | 78.54 | 79.73 | 86.61 | 30.19 | 14.08 |
| TerraMind-1.0-base | 84.96 | 70.65 | 82.45 | 83.81 | 81.40 | 89.51 | 97.60 | 14.81 |
| TerraMind-1.0-large | 86.06 | 72.32 | 83.58 | 84.44 | 82.88 | 90.29 | 317.86 | 24.45 |
Latency: mean over 100 runs, batch 1, 224 x 224, fp16 autocast, NVIDIA Tesla T4. Mean metrics (mIoU / F1 / Prec. / Rec. / mAP) are averaged over the 5 foreground classes (Clutter excluded), the usual ISPRS convention. OA uses all pixels.
| Model | Changed factor | Overrides | mIoU | Δ mIoU vs. base (pp) |
|---|---|---|---|---|
| TerraMind-1.0-base | - (reference) | - | 70.65 | +0.00 |
| KAO-TerraMind-A ⬅ | optimisation: stronger backbone adaptation | backbone_lr_mult=0.3, weight_decay=0.05 |
71.74 | +1.09 |
| KAO-TerraMind-B | loss: stronger overlap (Dice) term for small / rare classes (e.g. car) | dice_weight=1.0 |
70.79 | +0.14 |
| KAO-TerraMind-C | decoder: wider UperNet, deeper-biased feature taps, more dropout | decoder_channels=512, head_dropout=0.2, indices=[3, 6, 9, 11] |
70.96 | +0.31 |
| Class | KAO-A | KAO-B | KAO-C | TM-Tiny | TM-Small | TM-Base | TM-Large |
|---|---|---|---|---|---|---|---|
| 🛣️ Impervious surfaces | 88.10 | 87.26 | 87.51 | 84.80 | 84.16 | 87.34 | 88.52 |
| 🏢 Building | 91.39 | 90.44 | 90.42 | 87.38 | 87.50 | 90.06 | 91.50 |
| 🌱 Low vegetation | 78.13 | 77.60 | 77.12 | 75.18 | 74.96 | 77.61 | 78.77 |
| 🌳 Tree | 85.12 | 85.09 | 84.94 | 82.85 | 83.79 | 84.84 | 85.45 |
| 🚗 Car | 73.11 | 72.25 | 73.34 | 61.77 | 64.52 | 72.38 | 73.63 |
| 🗑️ Clutter | 44.57 | 41.06 | 43.68 | 40.00 | 44.36 | 44.22 | 39.78 |
| Class | KAO-A | KAO-B | KAO-C | TM-Tiny | TM-Small | TM-Base | TM-Large |
|---|---|---|---|---|---|---|---|
| 🛣️ Impervious surfaces | 78.73 | 77.40 | 77.79 | 73.61 | 72.65 | 77.53 | 79.41 |
| 🏢 Building | 84.14 | 82.54 | 82.51 | 77.59 | 77.78 | 81.92 | 84.34 |
| 🌱 Low vegetation | 64.11 | 63.40 | 62.76 | 60.23 | 59.94 | 63.41 | 64.98 |
| 🌳 Tree | 74.09 | 74.05 | 73.82 | 70.72 | 72.10 | 73.67 | 74.60 |
| 🚗 Car | 57.62 | 56.56 | 57.90 | 44.69 | 47.62 | 56.72 | 58.27 |
| 🗑️ Clutter | 28.68 | 25.83 | 27.94 | 25.00 | 28.50 | 28.39 | 24.83 |
pip install "terratorch>=1.2.4" "diffusers==0.30.0" huggingface_hub pillow tifffile numpy torch
# downloads kao_terramind_inference.py + terramind_v1_kaoA_best.pt from this repo
hf download kaopanboonyuen/KAO-TerraMind-A-Vaihingen --local-dir KAO-TerraMind-A-Vaihingen # older huggingface_hub: huggingface-cli download
cd KAO-TerraMind-A-Vaihingen
python kao_terramind_inference.py \
--input my_tile.tif \
--output prediction.png \
--device cuda
Use --ckpt path/to/terramind_v1_kaoA_best.pt for a local checkpoint, --overlap 56 for smoother large-scene mosaics, and --save-index to also write the raw class-index map.
from kao_terramind_inference import load_model, predict_image, colorize, read_image
net, ck = load_model("terramind_v1_kaoA_best.pt", device="cuda")
img = read_image("my_tile.tif") # (H, W, 3) uint8
pred = predict_image(net, img, ck, device="cuda") # (H, W) class ids, any image size
rgb = colorize(pred) # (H, W, 3) uint8 map in the dataset palette
📌 Input contract: aerial IRRG (NIR, R, G), 8-bit, fed to the RGB modality in stored band order, any size (processed in 224 x 224 windows with zero-padded borders, exactly like the evaluation protocol). Normalisation statistics are read from the checkpoint.
| File | Description |
|---|---|
📖 README.md |
This model card |
⚙️ config.json |
Architecture arguments, class map, normalisation, recipe and test metrics |
🐍 kao_terramind_inference.py |
Inference-only helper (checkpoint loading, tiled prediction, colour maps, CLI) |
🧠 terramind_v1_kaoA_best.pt |
Fine-tuned weights (backbone + decoder + architecture arguments) |
If you use this model, please cite:
@article{panboonyuen2027kaoterramind,
title = {KAO-TerraMind: Targeted Adaptation of Geospatial Foundation Models for Dense Prediction},
author = {Panboonyuen, Teerapong},
note = {Manuscript in preparation for submission to the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2027)},
year = {2027}
}
KAO-TerraMind builds on TerraMind; please also cite:
@inproceedings{jakubik2025terramind,
title = {Terramind: Large-scale generative multimodality for earth observation},
author = {Jakubik, Johannes and Yang, Felix and Blumenstiel, Benedikt and Scheurer, Erik and Sedona, Rocco and Maurogiovanni, Stefano and Bosmans, Jente and Dionelis, Nikolaos and Marsocci, Valerio and Kopp, Niklas and others},
booktitle = {2025 IEEE/CVF International Conference on Computer Vision (ICCV)},
pages = {7383--7394},
year = {2025},
organization = {IEEE}
}
🌍 TerraMind (IBM · ESA) · 🔥 TerraTorch · 🤗 Hugging Face · 🗂️ ISPRS Vaihingen dataset creators · 🇹🇭 Chulalongkorn University
Teerapong Panboonyuen, Ph.D. (Kao) Vice President of AI, Government Savings Bank C2F Postdoctoral Fellow, Chulalongkorn University Adjunct Professor, Khon Kaen University 🇹🇭 Thailand
Research interests: Geospatial AI · Earth Observation · Foundation Models · Remote Sensing · Dense Prediction
From pixels to representations. From representations to intelligence.
KAO-TerraMind-A-Vaihingen · CVPR 2027 reproducibility release