Automated Detection and Classification of Pavement Distresses using 3D Pavement Surface Images and Deep Learning

Abstract

Pavement distresses lead to pavement deterioration and failure. Accurate identification and classification of distresses helps agencies evaluate the condition of their pavement infrastructure and assists in decision-making processes on pavement maintenance and rehabilitation. The state of the art is automated pavement distress detection using vision-based methods. This study implements two deep learning techniques, Faster Region-based Convolutional Neural Networks (R-CNN) and You Only Look Once (YOLO) v3, for automated distress detection and classification of high resolution (1,800 × 1,200) three-dimensional (3D) asphalt and concrete pavement images. The training and validation dataset contained 625 images that included distresses manually annotated with bounding boxes representing the location and types of distresses and 798 no-distress images. Data augmentation was performed to enable more balanced representation of class labels and prevent overfitting. YOLO and Faster R-CNN achieved 89.8% and 89.6% accuracy respectively. Precision-recall curves were used to determine the average precision (AP), which is the area under the precision-recall curve. The AP values for YOLO and Faster R-CNN were 90.2% and 89.2% respectively, indicating strong performance for both models. Receiver operating characteristic (ROC) curves were also developed to determine the area under the curve, and the resulting area under the curve values of 0.96 for YOLO and 0.95 for Faster R-CNN also indicate robust performance. Finally, the models were evaluated by developing confusion matrices comparing our proposed model with manual quality assurance and quality control (QA/QC) results performed on automated pavement data. A very high level of match to manual QA/QC, namely 97.6% for YOLO and 96.9% for Faster R-CNN, suggest the proposed methodology has potential as a replacement for manual QA/QC.

Exploring automated object detection methods for manholes using classical computer vision and deep learning

“…In [15] (2021), two deep learning techniques were implemented for automated pavement distress detection and classification, namely Faster Region-based Convolutional Neural Networks (R-CNN) and YOLOv3. These deep learning frameworks were trained on a dataset the authors collected and validation accuracy was indicated using Average Precision (AP) and Receiver Operating Characteristic (ROC) curves.…”

Comparing Different Deep Learning Architectures as Vision-Based Multi-Label Classifiers for Identification of Multiple Distresses on Asphalt Pavement

“…The development of vision-based automated pavement distress identification methods is well documented in the literature. Methods like image processing (intensitythresholding, edge detection, and seed-based) and machine learning (unsupervised learning, supervised learning, and DL) have been widely used for distress identification and quantification (12)(13)(14)(15)(16)(17)(18)(19)(20).…”

Improvement of Multiclass Classification of Pavement Objects Using Intensity and Range Images

“…e 3D laser imaging technology has been used to evaluate crack [66,67], pothole [68], raveling [69], rutting [70], joint [71], and texture [72]. Ghosh et al [73] employed YOLO and Faster R-CNN to detect cracks in range images collected by the 3D imaging system. Yang et al [74] utilized 3D laser technology to measure the growth of crack lengths when they are sealed and non-sealed to quantify the crack sealing benefit.…”