Image-Based Pothole Detection Using Multi-Scale Feature Network and Risk Assessment

Abstract

Potholes on road surfaces pose a serious hazard to vehicles and passengers due to the difficulty detecting them and the short response time. Therefore, many government agencies are applying various pothole-detection algorithms for road maintenance. However, current methods based on object detection are unclear in terms of real-time detection when using low-spec hardware systems. In this study, the SPFPN-YOLOv4 tiny was developed by combining spatial pyramid pooling and feature pyramid network with CSPDarknet53-tiny. A total of 2665 datasets were obtained via data augmentation, such as gamma regulation, horizontal flip, and scaling to compensate for the lack of data, and were divided into training, validation, and test of 70%, 20%, and 10% ratios, respectively. As a result of the comparison of YOLOv2, YOLOv3, YOLOv4 tiny, and SPFPN-YOLOv4 tiny, the SPFPN-YOLOv4 tiny showed approximately 2–5% performance improvement in the mean average precision (intersection over union = 0.5). In addition, the risk assessment based on the proposed SPFPN-YOLOv4 tiny was calculated by comparing the tire contact patch size with pothole size by applying the pinhole camera and distance estimation equation. In conclusion, we developed an end-to-end algorithm that can detect potholes and classify the risks in real-time using 2D pothole images.

Cooperative Saliency-Based Pothole Detection and AR Rendering for Increased Situational Awareness

“…Heo et al [23] proposed a 2D pothole risk assessment standard to visually indicate risk signals to the driver by comparing the size of the pothole detected using the developed model with the size of the tire contact patch area. The authors conclude that if a risk assessment method could be used in real-time, it may not only be useful for road maintenance but also for detecting large potholes that are not recognizable by drivers in driving situations.…”

Integrating Sigmoid Calibration Function into Entropy Thresholding Segmentation for Enhanced Recognition of Potholes Imaged Using a UAV Multispectral Sensor

“…The responsibility of thresholding is to convert a projected probability or score into a certain class. Several studies on pothole characterization based on UAV multispectral imagery focused on improving methods and techniques for detecting potholes [18,36]. However, the deployment of these approaches without them being subjected to a calibration process poses a question regarding their ability to compensate for information loss due to noise associated with ultra-spatial resolution of UAV sensors.…”