plantvillage-resnet18

Plant leaf disease classifier for the capstone project AI-Powered Plant Disease Detection and Farmer Assistance (IIIT-H, Group 4). A farmer uploads a photo of a crop leaf; this model returns the most likely PlantVillage class and a confidence score. A LangGraph agent then explains the result and suggests treatment from a curated agricultural knowledge base.

• Base model: neghani/plantvillage-resnet18, fine-tuned end to end
• Classes: 38 (14 crops)
• Trained: 2026-11-04 · Hub id: neghani/plantvillage-resnet18

Intended use

• Input: a single RGB photo of one leaf, roughly centred, taken in daylight.
• Output: PlantVillage class name (for example Tomato___Early_blight) plus softmax probability.
• Treat a top-1 probability below 0.5 as "not sure" and ask for a clearer photo or expert confirmation (see threshold sweep below).
• Not a substitute for a plant pathologist. Treatment decisions must be confirmed with a local agricultural extension officer.

Training data

• PlantVillage color images, stratified per-class split with seed 42: train 43445 · validation 5430 · test 5430.
• Stage 2 (tag v1.1 and later) also uses Cropped-PlantDoc (Singh et al. 2020, CC BY 4.0): 27 of 38 classes; the other 11 stay lab-only. See run_info.json for the PlantDoc commit and data manifest.
• Augmentation (phone-photo robustness): random resized crop (70-100% PlantVillage, 50-100% PlantDoc), horizontal and vertical flip, rotation up to 180°, brightness/contrast/saturation jitter, random Gaussian blur.
• Evaluation preprocessing: resize shortest edge, centre crop 224, normalise (same as the bundled preprocessor_config.json).

Results on the held-out PlantVillage test split

Confusion matrix: reports/confusion_matrix.png · Full per-class report: reports/classification_report.txt

Weaker classes

The weakest PlantVillage test classes are Potato healthy and Corn Cercospora.

Confusion matrix: reports/confusion_matrix.png.

Per-class precision / recall / F1

Confidence threshold sweep

Coverage is the share of test images with top-1 probability at or above the threshold; the remaining images would be answered with "not sure". Recommended CONFIDENCE_THRESHOLD for the API: 0.5 (lowest threshold whose accuracy on covered images is at least 97% with coverage of at least 50%).

Real-world images (field and web photos)

PlantVillage test metrics above are in-distribution (lab-style photos). Phone photos with clutter, blur, and mixed backgrounds score lower. After PlantDoc stage-2 fine-tune (tag v1.1), run model/final_test.py and update this section from model/final_report.md.

For tag v1.2 (PlantVillage stage 1 only): end-to-end real-world numbers are not recorded yet — publish v1.1 after stage-2 training and calibration.

Real-world results before and after fine-tuning

Same real-world set, live-app chain (gate + strategy B). Each cell is passes/n (rate), 95% Wilson CI.

Calibration

Temperature scaling fitted on PlantDoc validation (15 bins).

Reliability (before and after on the same axes):

Latency

The gap is the app path (quality gate, crop, HTTP), not the forward pass.

Crop strategy comparison (A–E)

Tune-split comparison from eval_strategies.py. Each strategy at its own best threshold.

Limitations

• PlantVillage images are lab-style photos on plain backgrounds; accuracy on field photos is lower. The real-world section above reports the gap.
• Only the 38 PlantVillage classes are known. Other crops, other diseases, nutrient deficiencies and pest damage will be forced into the nearest known class; rely on the threshold and the "not sure" path.
• One leaf per photo. Mixed infections and whole-plant shots are out of scope.
• Confidence is a softmax probability. Stage 2 ships a fitted temperature in calibration.json (NLL on PlantDoc validation). Treat remaining overconfidence as a limitation.
• PlantDoc and the 50-image real-world set are web photos; they overstate phone-photo performance. The one-shot comparison is model/final_report.md (not used for any choice).

Training details

• base_model: neghani/plantvillage-resnet18
• base_revision: v1.0
• init_from: neghani/plantvillage-resnet18
• extra_data: {'train': 1967, 'validation': 302, 'skipped_labels': {}, 'dir': '/Users/ganeshkumar/Projects/iit_program/iiith-group4-capstone-plat-leaf-decease-detection/model/data/plantdoc_train', 'fraction': 0.5, 'epoch_samples': 4000}
• plantdoc_fraction: 0.5
• epoch_samples: 4000
• plantdoc_commit: 5467f6012d78d1c446145d5f582da6096f852ae8
• plantdoc_commit_pinned: 5467f6012d78d1c446145d5f582da6096f852ae8
• data_manifest: {'plantdoc_commit': '5467f6012d78d1c446145d5f582da6096f852ae8', 'plantdoc_commit_pinned': '5467f6012d78d1c446145d5f582da6096f852ae8', 'commit_matches_pin': True, 'params': {'phash_dist': 6, 'plantdoc_fraction': 0.5, 'epoch_samples': 4000}, 'counts': {'plantdoc_train': 1967, 'plantdoc_validation': 302, 'final_test': 284, 'plantvillage_train': 43445}, 'counts_by_source': {'plantdoc_train': {'plantdoc': 1967}, 'plantdoc_validation': {'plantdoc': 302}, 'final_test': {'plantdoc': 236, 'real_world_set': 48}, 'plantvillage_train': {'plantvillage': 43445}}, 'counts_by_stratum': {'final_test': {'plantdoc_test': 236, 'real_world_set': 48}}, 'per_class': {'Apple___Apple_scab': {'plantdoc_train': 74, 'plantdoc_validation': 7, 'final_test': 12, 'plantvillage_train': 504}, 'Apple___Black_rot': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 497}, 'Apple___Cedar_apple_rust': {'plantdoc_train': 69, 'plantdoc_validation': 8, 'final_test': 11, 'plantvillage_train': 219}, 'Apple___healthy': {'plantdoc_train': 64, 'plantdoc_validation': 16, 'final_test': 11, 'plantvillage_train': 1317}, 'Blueberry___healthy': {'plantdoc_train': 89, 'plantdoc_validation': 12, 'final_test': 13, 'plantvillage_train': 1202}, 'Cherry_(including_sour)___Powdery_mildew': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 842}, 'Cherry_(including_sour)___healthy': {'plantdoc_train': 37, 'plantdoc_validation': 8, 'final_test': 12, 'plantvillage_train': 684}, 'Corn_(maize)___Cercospora_leaf_spot Gray_leaf_spot': {'plantdoc_train': 48, 'plantdoc_validation': 13, 'final_test': 5, 'plantvillage_train': 411}, 'Corn_(maize)___Common_rust_': {'plantdoc_train': 91, 'plantdoc_validation': 13, 'final_test': 12, 'plantvillage_train': 954}, 'Corn_(maize)___Northern_Leaf_Blight': {'plantdoc_train': 152, 'plantdoc_validation': 25, 'final_test': 13, 'plantvillage_train': 789}, 'Corn_(maize)___healthy': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 930}, 'Grape___Black_rot': {'plantdoc_train': 52, 'plantdoc_validation': 2, 'final_test': 10, 'plantvillage_train': 944}, 'Grape___Esca_(Black_Measles)': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 1107}, 'Grape___Leaf_blight_(Isariopsis_Leaf_Spot)': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 860}, 'Grape___healthy': {'plantdoc_train': 50, 'plantdoc_validation': 4, 'final_test': 14, 'plantvillage_train': 339}, 'Orange___Haunglongbing_(Citrus_greening)': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 4405}, 'Peach___Bacterial_spot': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 1837}, 'Peach___healthy': {'plantdoc_train': 81, 'plantdoc_validation': 19, 'final_test': 11, 'plantvillage_train': 288}, 'Pepper,_bell___Bacterial_spot': {'plantdoc_train': 55, 'plantdoc_validation': 4, 'final_test': 11, 'plantvillage_train': 797}, 'Pepper,_bell___healthy': {'plantdoc_train': 43, 'plantdoc_validation': 8, 'final_test': 10, 'plantvillage_train': 1182}, 'Potato___Early_blight': {'plantdoc_train': 85, 'plantdoc_validation': 18, 'final_test': 10, 'plantvillage_train': 800}, 'Potato___Late_blight': {'plantdoc_train': 85, 'plantdoc_validation': 10, 'final_test': 9, 'plantvillage_train': 800}, 'Potato___healthy': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 122}, 'Raspberry___healthy': {'plantdoc_train': 94, 'plantdoc_validation': 16, 'final_test': 9, 'plantvillage_train': 297}, 'Soybean___healthy': {'plantdoc_train': 50, 'plantdoc_validation': 5, 'final_test': 10, 'plantvillage_train': 4072}, 'Squash___Powdery_mildew': {'plantdoc_train': 106, 'plantdoc_validation': 16, 'final_test': 8, 'plantvillage_train': 1467}, 'Strawberry___Leaf_scorch': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 887}, 'Strawberry___healthy': {'plantdoc_train': 79, 'plantdoc_validation': 7, 'final_test': 10, 'plantvillage_train': 364}, 'Tomato___Bacterial_spot': {'plantdoc_train': 88, 'plantdoc_validation': 10, 'final_test': 11, 'plantvillage_train': 1701}, 'Tomato___Early_blight': {'plantdoc_train': 65, 'plantdoc_validation': 11, 'final_test': 11, 'plantvillage_train': 800}, 'Tomato___Late_blight': {'plantdoc_train': 83, 'plantdoc_validation': 15, 'final_test': 11, 'plantvillage_train': 1527}, 'Tomato___Leaf_Mold': {'plantdoc_train': 70, 'plantdoc_validation': 13, 'final_test': 8, 'plantvillage_train': 762}, 'Tomato___Septoria_leaf_spot': {'plantdoc_train': 115, 'plantdoc_validation': 21, 'final_test': 13, 'plantvillage_train': 1417}, 'Tomato___Spider_mites Two-spotted_spider_mite': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 1340}, 'Tomato___Target_Spot': {'plantdoc_train': 0, 'plantdoc_validation': 0, 'final_test': 0, 'plantvillage_train': 1124}, 'Tomato___Tomato_Yellow_Leaf_Curl_Virus': {'plantdoc_train': 58, 'plantdoc_validation': 10, 'final_test': 7, 'plantvillage_train': 4285}, 'Tomato___Tomato_mosaic_virus': {'plantdoc_train': 39, 'plantdoc_validation': 3, 'final_test': 12, 'plantvillage_train': 299}, 'Tomato___healthy': {'plantdoc_train': 45, 'plantdoc_validation': 8, 'final_test': 10, 'plantvillage_train': 1273}}, 'exclusions': {'near_dup_of_plantdoc_test': 16, 'near_dup_of_real_world_set': 1, 'path_in_real_world_set': 48, 'spider_mites_class': 2}, 'plantvillage_counts': {'train': 43445, 'validation': 5430, 'test': 5430}}
• code_commit: 4eaadca4374d008ec89662c6a4da143b74ac0fde
• code_dirty: True
• augmentation: {'policy': 'default', 'pv_rrc_scale': [0.7, 1.0], 'pd_rrc_scale': [0.5, 1.0], 'ops': ['RandomResizedCrop', 'RandomHorizontalFlip', 'RandomVerticalFlip', 'RandomRotation', 'ColorJitter', 'RandomApply', 'ToTensor', 'Normalize']}
• label_smoothing_factor: 0.1
• train_rows: 45412
• num_labels: 38
• epochs: 8.0
• learning_rate: 0.0001
• per_device_train_batch_size: 32
• weight_decay: 0.01
• warmup_steps: 0
• lr_scheduler_type: SchedulerType.COSINE
• fp16: False
• bf16: False
• dataloader_num_workers: 4
• dataloader_pin_memory: True
• seed: 42
• wall_time_sec: 483.1
• device: mps

Validation by epoch

Rising validation loss in later epochs is the overfitting check.

Use in the API (Phase 5)

from transformers import pipeline

clf = pipeline("image-classification", model="neghani/plantvillage-resnet18", revision="v1.2")
preds = clf("leaf.jpg", top_k=3)
# [{'label': 'Tomato___Early_blight', 'score': 0.93}, ...]
top = preds[0]
if top["score"] < 0.5:
    print("Not sure - please retake the photo or consult an expert")

Class names match the keys of api/data/knowledge/diseases.json in the project repository, so the prediction can be looked up directly for symptoms and treatment guidance.

Project repository: https://github.com/neghani/iiith-group4-capstone-plat-leaf-decease-detection

Disclaimer

Treatment suggestions produced from this model's output are general guidance only and must be validated with local agricultural experts before critical action.