Weed Swin: per-image median adaptive training crops
Copy of haiquanli/weed_swin at revision 4d54d461cd8358861fdf741a15366990f69dd111, updated with the best validation-mask checkpoint (epoch 32) from one 42-epoch run. The original repository is unchanged.
Four-class Swin-S Mask R-CNN: 0=weed, 1=lettuce, 2=spinach_seedling, 3=spinach_growth. Training grid selection uses per-photo median annotated object size, target object size 128 pixels, maximum crop factor 4, and minimum crop edge 512. Inference uses whole photographs without requiring object annotations. The same split, sampling budget, optimizer and class balancing were used for the fixed-grid comparison.
Whole-image test results
126 held-out original photos, 1,270 annotated instances. COCO AP uses IoU 0.50:0.05:0.95, score threshold 0.001 and maxDets [1,10,100], without additional mask NMS.
| Metric | Fixed grids | Median grids |
|---|---|---|
| Box mAP | 53.30% | 54.08% |
| Box AP50 | 75.74% | 74.44% |
| Mask mAP | 47.65% | 49.05% |
| Mask AP50 | 72.61% | 74.21% |
| Class | Mask AP | Mask AP50 |
|---|---|---|
| Weed | 35.33% | 57.47% |
| Lettuce | 82.24% | 95.71% |
| Spinach seedling | 32.27% | 64.10% |
| Growth-stage spinach | 46.36% | 79.57% |
Weed annotations may be incomplete, which can penalize detections of unlabeled weeds. This is a single-seed experiment. Results from the source repository used crop-level evaluation and are not directly comparable to these whole-image results. Full metrics and comparison are in evaluation/.
Usage
Use the MMDetection 3.x environment described in requirements.txt, with MMCV 2.1.0, MMEngine, PyTorch and Transformers. This repository contains custom Python model code.
from PIL import Image, ImageOps
from transformers import AutoImageProcessor, AutoModelForObjectDetection
import torch
repo = "haiquanli/weed_swin_dynamic_crop"
model = AutoModelForObjectDetection.from_pretrained(repo, trust_remote_code=True)
processor = AutoImageProcessor.from_pretrained(repo, trust_remote_code=True)
image = ImageOps.exif_transpose(Image.open("plants.jpg")).convert("RGB")
inputs = processor(images=image, return_tensors="pt")
inputs["pixel_values"] = inputs["pixel_values"].to(model.device)
with torch.inference_mode():
outputs = model(**inputs, score_threshold=0.25)
predictions = processor.post_process_object_detection(outputs, threshold=0.25)[0]
The processor preserves aspect ratio and passes raw BGR pixels to the MMDetection preprocessor. Outputs include boxes, masks, scores and contiguous class labels in original image coordinates. COCO category IDs for evaluation are [1,2,4,5]. Training crop plans are provenance only, not an inference-time tiling policy.