RoadPulse pothole classifier (Core ML)

On-device pothole / normal-road classifier behind RoadPulse, a SwiftUI iOS app (iOS 17+) that photographs road hazards, classifies them on the phone, geotags them and maps them.

Test accuracy: 96.7% on 211 held-out images · MobileNetV3-Small · 3.1 MB · runs fully offline.

Files

Use in an iOS app (Vision)

import CoreML
import Vision

let config = MLModelConfiguration()
let compiled = try MLModel(contentsOf: Bundle.main.url(forResource: "RoadClassifier", withExtension: "mlmodelc")!,
                           configuration: config)
let request = VNCoreMLRequest(model: try VNCoreMLModel(for: compiled)) { request, _ in
    guard let top = (request.results as? [VNClassificationObservation])?.first else { return }
    print(top.identifier, top.confidence) // "pothole" 0.97
}
request.imageCropAndScaleOption = .scaleFill // matches the training resize
try VNImageRequestHandler(cgImage: cgImage, orientation: .up).perform([request])

Add the .mlpackage to an Xcode target; Xcode compiles it to RoadClassifier.mlmodelc. In RoadPulse the request runs off the main thread (Services/ClassifierService.swift), and the confidence also pre-fills the hazard's severity (≥ 0.95 → 5, ≥ 0.85 → 4, ≥ 0.70 → 3, else 2).

Verified in the app's CI

The RoadPulse XCTests run this exact model in the iOS simulator on every push (GitHub Actions, Xcode 16.4):

• three held-out pothole photos (never used in training) are classified as pothole with ≥ 80% confidence;
• two held-out normal-road photos are classified as normal;
• a photo still classifies correctly after the app's resize-and-JPEG preparation step.

Intended use and limits

A student project (Honours course iOS Application Development using SwiftUI) for flagging likely potholes in a road-reporting app. A person always confirms the report. Do not use it for safety-critical decisions. It knows two classes only: cracks, waterlogging and debris are chosen by the user in the app, not detected.

Training code: ml/ in the GitHub repo (download_data.py, train.py, convert.py).

---

Test accuracy: 96.7% on 211 held-out images (target ≥ 88%).

Model

• MobileNetV3-Small (ImageNet weights), classifier head replaced with 2 outputs, full fine-tune.
• Classes: normal, pothole. No crack class: no suitable close-up crack dataset was found.
• Input 224×224 RGB, ImageNet mean (0.485, 0.456, 0.406), single std 0.226 (what Core ML's ImageType scale/bias can express). Use imageCropAndScaleOption = .scaleFill in Vision.
• 20 epochs, batch 64, AdamW lr 0.001, cosine schedule, AMP, class-weighted cross-entropy (label smoothing 0.05). Trained in 1.2 min on an RTX 3050 Ti. Best epoch picked on validation accuracy.

Data (all public, downloaded by ml/download_data.py, no Kaggle key)

Split 80/10/10 by source photo (augmented copies never straddle splits), stratified per source and class: train 1717, val 210, test 211.

Confusion matrix (test; rows = actual, columns = predicted)

Test accuracy by source and class

Caveats (read before trusting the headline number)

• Each source is skewed: rupesh is pothole-only and grayscale, pave is normal-only and low-resolution. A model can partly learn the source. Training uses random grayscale/blur to weaken those shortcuts, but the in-domain rows above (taroii/*, arpit/*) are the most honest signal.
• The datasets are small (about 2k images) and mostly European/Asian roads. Expect lower accuracy on Mumbai monsoon photos; test with your own phone photos before relying on it.
• normal includes wide street scenes and flat pavement patches, not only close-ups.

Training curve (val accuracy per epoch)

0.93 0.97 0.93 0.91 0.94 0.97 0.97 0.95 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98 0.98