🌱 Sarox AgriDetector

Crop, plant part and condition from a photo, powered by Sarox Plangton 2.0: a small image model (4.77 M parameters, 19.5 MB) that recognises 133 conditions across 26 crops and runs on an ordinary CPU or inside a web browser.

▶ Try it: Sarox AgriDetector Space. It runs in your browser, so your photos are never uploaded.

Features

• 🌾 26 crops, 133 conditions, 6 plant parts. Diseases, pests and healthy plants on leaves (102 classes), fruit, flowers and ears, stems and trunks, wheat roots and whole plants.
• 🪴 Whole-plant check. Combine up to 6 photos of one plant (say a leaf, the stem and a fruit) into one plant-level answer: the crop, the chance the plant is healthy and the likely problems.
• 🧭 Knows when it doesn't know. An unknown-photo gate and a calibrated "not sure" rule: 99.7% of unrelated test images (people, objects, textures, soil) got no answer, and only 1.76% of the answers it did give were wrong.
• 🎯 Honest confidence. Temperature-scaled probabilities (calibration error 0.006) and a 95% "possible conditions" set for every photo.
• ⚡ Small and fast. One 19.5 MB ONNX file, no PyTorch and no GPU; about 7–35 ms per photo on a laptop CPU.
• 🔒 Private demo. The Space runs the model with ONNX Runtime Web on the visitor's own device.
• ✅ Re-checked after export. The uploaded model.onnx and inference.py reproduce the training results on rebuilt test photos (see below).

Accuracy

Held-out test set, evaluated once after every training choice was fixed: 13,882 labelled photos never seen in training (10,787 field photos, 3,095 lab photos), plus 664 unrelated images for the unknown-photo check.

By plant part (accuracy / macro-F1 / test photos): leaf 94.6% / 93.3% / 12,085 · fruit 92.5% / 91.1% / 265 · flower 94.8% / 96.2% / 542 · stem 97.0% / 96.6% / 607 · root 70.1% / 82.4% / 87 · whole plant 97.3% / 95.3% / 296.

By crop

Weakest classes (F1, test photos): wheat tan spot 52.9% (102) · apple scab 62.1% (14) · soybean healthy 66.7% (10) · wheat leaf blight 68.3% (93) · apple cedar rust 71.0% (16) · wheat stem rust 71.8% (50) · wheat common root rot 74.9% (87). Precision, recall and F1 for all 133 classes are in metrics.json.

Re-check of the uploaded files. The test photos of three public datasets were rebuilt from the training split list and run through model.onnx + inference.py: the 3,095-photo v1 test set scored 98.77% accuracy / 98.79% macro-F1 (the training notebook measured 98.71% / 98.72% in PyTorch; merging layers for export moves a few borderline photos), and the per-class results of the five sugarcane and chilli classes that come from a single dataset matched the notebook exactly. The browser demo gives the same answers as inference.py.

How to use

pip install -r requirements.txt huggingface_hub   # onnxruntime, numpy, pillow (Python 3.11–3.14)

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("sarang-sketch/sarox-agriDetect", allow_patterns=["inference.py", "config.json", "model.onnx"])
sys.path.insert(0, path)
from inference import assess_plant, load_model, predict_image

model = load_model(path)
print(predict_image("leaf.jpg", model))                                # one photo (path, bytes or PIL image)
print(assess_plant(["leaf.jpg", "stem.jpg", "fruit.jpg"], model))      # several photos of ONE plant

In production, pin a version with snapshot_download(..., revision="<commit hash>"). From a terminal: python inference.py photo.jpg [more.jpg ...]. Unreadable files, images over 40 megapixels and images more than 20 times wider than tall (or the reverse) raise InvalidImageError; map it to HTTP 400 in an API.

predict_image returns a status:

• success: predicted_class, display_name, crop, organ, condition, is_healthy and, for three classes with weak training data, low_trust_reason;
• uncertain: confidence below the abstention threshold, so no answer is given;
• unknown: the photo does not look like the training data (another plant, a person, soil, a document…).

Every result also has the calibrated confidence, top_predictions (3), possible_conditions_95 (a set that contains the true class about 95% of the time for photos like the test data), crop_probabilities, organ_probabilities, healthy_probability and energy. assess_plant adds a plant-level crop, crop_confidence, photo_agreement, plant_healthy_probability and possible_problems, and returns inconsistent when the photos look like different crops.

How it decides

1. Calibration: probabilities = softmax(logits / T) with T = 1.4106 (temperature scaling, fitted on validation).
2. Unknown-photo gate: energy E = −T·log Σ exp(zₖ/T). Above −9.725 (the 95th percentile of validation plant photos) the photo is unknown. By design this also turns away about 5% of genuine plant photos, so ask users to retake those.
3. Abstention: answer only when the calibrated confidence is ≥ 0.80. The threshold was chosen by fixed-sequence testing on held-out calibration data so that the error of answered photos is ≤ 2% with 95% confidence (Clopper–Pearson); on the test set it was 1.76%.
4. Conformal set (α = 0.05): every class with probability ≥ 0.2769.
5. Several photos of one plant: the crop is the product of the photos' crop probabilities (agreement = the sum of that product), each problem is combined with a noisy-OR, and the plant counts as healthy only if every photo does.

Files

To keep the repository small, two training by-products are not uploaded: an INT8 copy of the model (5.6 MB, but it changes 4.4% of the answers) and the 6.9 MB list of image hashes that defines the exact data split.

Preprocessing (identical in training evaluation, inference.py and the browser demo): EXIF orientation, RGB with transparent pixels on white, shorter side to 256 (bilinear), centre crop 224, ImageNet mean/std, float32 NCHW.

Training

• Backbone: RepViT-M0.9 (timm/repvit_m0_9.dist_450e_in1k, ImageNet-1k), 224 × 224 input.
• Data: 22 public Kaggle datasets (listed below): 63,467 training, 13,707 validation and 14,198 test images, plus 4,429 unrelated images (Caltech-101, textures, soil) split the same way: 3,101 for outlier exposure in training, 664 for the test.
• Split: 70/15/15 per class, made before any augmentation. Near-duplicates (same dHash) and augmented copies of one photo stay in one split. The Sarox Plangton 1.0 split was rebuilt exactly (fingerprint e42d5f03e46f360f), so no v1 test photo was used for training.
• Loss: logit-adjusted set likelihood (class imbalance, and partial labels such as "tomato leaf blight" = early or late blight) + 0.5 × outlier exposure. AdamW, weight decay 0.05, batch 64, seed 42.
• Schedule: 3 epochs training the heads, then 22 epochs training everything with a smaller backbone learning rate; the best validation macro-F1 (92.92%) was at epoch 25. Training took about 1.9 hours on one Tesla T4 (Google Colab).
• Export: ONNX opset 20, with RepViT's training-time branches and BatchNorm merged into single convolutions (largest logit change 1.7 × 10⁻⁴).

Limitations

• It only knows the 133 trained classes. A confident answer is not a diagnosis, and it gives no treatment or pesticide advice.
• Root data is tiny and noisy (wheat common root rot only), so root results are unreliable. Flower and stem classes exist for a few crops only and have no healthy examples, so the model cannot confirm that a stem, flower or root is healthy.
• Several sources come from Bangladesh, Ghana and other countries; Indian field photos may score lower. Crops with very few test photos (cherry, soybean, blueberry, apple) have uncertain scores.
• The statistical guarantees (conformal coverage, abstention error) hold for photos similar to the validation data, not for every field photo.

Data and licences

The model weights and code are released under Apache-2.0. The training images come from these datasets, each under its own licence as stated on its Kaggle page; several are CC BY 4.0 or CC BY-SA 4.0, one is ODbL, and the PlantVillage copy states no licence. Check them before commercial use.

emmarex/plantdisease (PlantVillage images; licence unknown) · imbikramsaha/paddy-doctor (CC0) · kushagra3204/wheat-plant-diseases (CC0) · kamal01/top-agriculture-crop-disease (CC0) · nirmalsankalana/crop-pest-and-disease-detection (CC0) · nirmalsankalana/plantdoc-dataset (CC0; PlantDoc, Singh et al. 2020) · rizwan123456789/potato-disease-leaf-datasetpld (ODbL) · sadikmahir/tomato-fruit-disease (Apache-2.0; Roboflow CC BY 4.0) · nirmalsankalana/sugarcane-leaf-disease-dataset (CC0) · nguynphancminh/cotton-leaf-diseases-dataset (Apache-2.0) · janmejaybhoi/cotton-disease-dataset (CC0) · nagabushan/coconut-tree-disease-dataset (CC BY-SA 4.0) · prudhvi143413s/anthracnose-disease-in-chilli-mobile-captured (CC0) · omkarpatil6o/crop-groundnut-disease (CC0) · shifatearman/bananalsd (CC BY-SA 4.0) · sujaykapadnis/banana-disease-recognition-dataset (CC BY 4.0) · warcoder/mango-leaf-disease-dataset (CC BY 4.0; MangoLeafBD) · niharipriya/citrus-plant-disease-image-dataset (CC BY-SA 4.0) · sujaykapadnis/eggplant-disease-recognition-dataset (CC BY 4.0) · imbikramsaha/caltech-101 (CC0) · shuvokumarbasak2030/soil-environment-analysis-dataset (MIT) · atamansemerci/describable-textures-dataset-dtd (MIT).

Methods referenced

RepViT (Wang et al., CVPR 2024) · logit adjustment (Menon et al., ICLR 2021) · outlier exposure (Hendrycks et al., ICLR 2019) · energy-based out-of-distribution detection (Liu et al., NeurIPS 2020) · temperature scaling (Guo et al., ICML 2017) · split conformal prediction (Angelopoulos & Bates, 2021) · Learn-then-Test risk control (Angelopoulos et al., 2021).