Automated pavement crack detection and segmentation based on two‐step convolutional neural network

Abstract

Cracking is a common pavement distress that would cause further severe problems if not repaired timely, which means that it is important to accurately extract the information of pavement cracks through detection and segmentation. Automated pavement crack detection and segmentation using deep learning are more efficient and accurate than conventional methods, which could be further improved. While many existing studies have utilized deep learning in pavement crack segmentation, which segments cracks from non-crack regions, few studies have taken the exact pavement crack detection into account, which identifies cracks in the images from other objects. A two-step pavement crack detection and segmentation method based on convolutional neural network was proposed in this paper. An automated pavement crack detection algorithm was developed using the modified You Only Look Once 3rd version in the first step. The proposed crack segmentation method in the second step was based on the modified U-Net, whose encoder was replaced with a pre-trained ResNet-34 and the upsample part was added with spatial and channel squeeze and excitation (SCSE) modules. Proposed method combines pavement crack detection and segmentation together, so that the detected cracks from the first step are segmented in the second step to improve the accuracy. A dataset of pavement crack images in different circumstances were also built for the study. The F1 score of proposed crack detection and segmentation methods are 90.58% and 95.75%, respectively, which are higher than other state-of-the-art methods. Compared with existing one-step pavement crack detection or segmentation methods, proposed two-step method showed advantages of accuracy.

Automatic Detection of Cracks in Asphalt Pavement Using Deep Learning to Overcome Weaknesses in Images and GIS Visualization

“…A variety of forms of ResNet have been proposed, such as ResNet18, ResNet34, ResNet50, ResNet101, ResNet152, among others, which are different in the number of layers and the number of parameters that can be learned. ResNet is so powerful that there are several examples of its use in pavement crack detection, as shown in [31,32]. In this study, ResNet50 is used in consideration of the balance between accuracy and computation time.…”

Automated Pavement Distress Detection, Classification and Measurement: A Review

“…We can make the dataset more exhaustive by applying the "Data Augmentation" method to the images collected. J. Liu [15] used image rotation (90•, 180•, 270•) and flipping (180• horizontally and vertically) to get 7104 images from 1184 initially captured.…”

Graph neural network and reinforcement learning for multi‐agent cooperative control of connected autonomous vehicles

“…In addition, energy‐based neural networks including restricted Boltzmann machine and conditional random fields have been applied in road structure distress classification (Maeda et al., 2019), concrete strength estimation (Rafiei et al., 2017), group worker activity recognition (Luo et al., 2020), real estate price estimation (Rafiei & Adeli, 2016), earthquake warning systems (Rafiei & Adeli, 2017a), and construction cost estimation (Rafiei & Adeli, 2018). Recently, more advanced neural networks including convolutional neural network (CNN) and recurrent neural network have been applied in the automatic pavement crack detection (Bang et al., 2019; A. Zhang, Wang, et al., 2019), and structural concrete damage detection (Liu et al., 2020; S. Li, Zhao, et al., 2019; Tong et al., 2020; X. Yang, Li, et al., 2018), and pedestrian counting (Shen et al., 2019). Other application contexts of DL—traffic flow prediction and traffic incident detection—were proposed initially by Adeli et al.…”