RIIAnet: A Real-Time Segmentation Network Integrated with Multi-Type Features of Different Depths for Pavement Cracks

Abstract

Considerable road mileage puts tremendous pressure on pavement crack detection and maintenance. In practice, using a small parameter model for fast and accurate image-based crack segmentation is a challenge. However, current mainstream convolutional neural networks allocate computing resources to the same type of operators, which ignores the impact of different levels of feature extractors on the model performance. In this research, an end-to-end real-time pavement crack segmentation network (RIIAnet) is designed to improve performance by deploying different types of operators in separate layers of the network structure. Based on the extraction characteristics of cracks by convolution, involution, and asymmetric convolution, in the shallow layers the crack segmentation task is matched to extract rich low-level features by the designed asymmetric convolution enhancement module (ACE). Meanwhile, in the deep layers, the designed residual expanded involution module (REI) is used to enhance the high-level semantic features. Furthermore, the existing involution operator that fails to converge during training is improved. The ablation experiment demonstrates that the optimal ratio of the convolution and REI is 1/3 to obtain the optimal resource allocation and ACE improves the performance of the model. Especially compared with seven classical deep learning models of different structures, the results show that the proposed model reaches the highest MIOU, MPA, Recall, and F1 score of 0.7705, 0.9868, 0.8047, and 0.8485, respectively. More importantly, the parameter size of the proposed model is dramatically reduced, which is 0.04 times that of U-Net. In practice, the proposed model can be implemented in images with a high resolution of 2048 × 1024 in real time.

Amphibious detection system for drainage pipes base on deep learning

“…In this paper, we replace some CBL [8] structures in the feature extraction network of YOlOV4-Tiny with the REI structure [9], which can utilize the spatial specificity of the involution operator [10] to strengthen the deep features. The algorithm structure is shown in Figure 3.…”

Analysis of segmentation performance of U-shape neural network algorithm by multi-type encoders for pavement cracks

“…But since the kernels learned by convolution have distributed eigenvalues, the direct conversion will result in significant information loss. In addition, [8] demonstrated by visualization that due to the special field of view threshold of asymmetric convolution, 1×d asymmetric convolution is sensitive to lateral cracks, and d×1 asymmetric convolution is more effective for vertical feature extraction. Therefore, in this paper, we use the structure as shown in Figure 1(c) to add 1×3 and 3×1 asymmetric convolution.…”

MFPNet: using multi-type features parallelism in deep layers to improve segmentation performance for pavement cracks

“…The involution-G operator [9] is an optimized Involution [10] operator that uses the self-normalization characteristics of group normalization [11] to avoid the problem that the algorithm cannot converge under conventional mini-batch training. The involution-G operator has the opposite characteristics of convolution, and its long-distance capture capability allows more spatial feature expressions at different levels.…”

“…The involution-G operator has the opposite characteristics of convolution, and its long-distance capture capability allows more spatial feature expressions at different levels. Meanwhile, experiments in [9] show that the inference speed of the involution-G operator is more sensitive to the feature map size than the 7×7 convolution operator of the same size. This paper compares the inference speed of two operators of different sizes in Section 4.2.…”

“…The dataset used in this paper is collected by the authors of [9] near Zhengzhou University, including 5110 images with a resolution of 416×416, including 1984 images for asphalt, 944 images for concrete, 1076 images for tar, and 1108 images for gravel. We randomly divided the collected images into the training set, validation set, and test set according to the ratio of 6:2:2.…”