Post‐earthquake damage recognition and condition assessment of bridges using UAV integrated with deep learning approach

Abstract

Rapid and accurate assessment of the damage to bridge structures after an earthquake can provide a basis for decision-making regarding post-earthquake emergency work. However, the traditional structural damage inspection techniques are subjective, time-consuming, and inefficient. This paper proposed a framework for rapid post-earthquake structural damage inspection and condition assessment by integrating the technologies of satellite, unmanned aerial vehicle (UAV), and smartphone with the deep learning approach. The images of structural components of post-earthquake bridges can be obtained by UAVs and smartphones. Furthermore, the multi-task high-resolution net (MT-HRNet) model was adopted to recognize the structural components and damage conditions by weighting and combining the loss functions of a single-task HRNet model. The performance of the proposed MT-HRNet model and the single-task HRNet model was verified based on the Tokaido dataset, which includes 2000 images of post-earthquake bridges. The results showed that the MT-HRNet model and the HRNet model exhibited equivalent recognition accuracy, while the number of floating-point-operations (FLOPs) and the parameters of the MT-HRNet model were reduced by 46.48% and 49.58% compared with the HRNet model. In addition, a method for the determination of the safety risk level of the post-earthquake bridge structures was developed, and the evaluation indices were established by considering the damage type, the spalling area, and the width of cracks as well as the recognition statistics of all images in Tokaido dataset. This study will provide a valuable reference for the rapid determination of structural safety level and the corresponding treatment measures of post-earthquake bridges.

A novel transformer-based semantic segmentation framework for structural condition assessment

“…Similarly, our model is compared with the three state-of-the-art models, namely an FCN 44 network, an ensembled model 55 and the MT-HRNet. 56 The damage dataset is composed of three classes, with the ''Non-damage'' class accounting for the majority of pixels (Table 1). It can be seen that using Swin Transformer as the backbone significantly improves the performance (8%), which is aligned with the results using MaskFormer.…”

“…Once the training is done, the trained model is applied to the validation set for the final evaluation. The IoU is compared with three state-of-the-art models, that is, the FCN58 network reported in the original Tokaido dataset paper, 44 the best version of ensembled models proposed by Liu et al, 55 and the multi-task high-resolution net (MT-HRNet: composed of multiple ResNet architectures) proposed by Ye et al 56 . Table 2 shows the performance of the structural component recognition task on the validation set.…”

“…Besides, the pixel accuracy (Acc) and mAcc values are presented for further demonstration. The mIoU is compared with the MT-HRNet since Ye et al 56 only reported the mean IoU across all classes. Our model achieved around 97% mIoU for component recognition, which is a significant improvement compared to 87.9%, 83.6%, and 71.9% as reported by Narazaki et al, 44 Liu et al, 55 and Ye et al, 56 respectively.…”

“…The mIoU is compared with the MT-HRNet since Ye et al 56 only reported the mean IoU across all classes. Our model achieved around 97% mIoU for component recognition, which is a significant improvement compared to 87.9%, 83.6%, and 71.9% as reported by Narazaki et al, 44 Liu et al, 55 and Ye et al, 56 respectively. Interestingly, the per-class metrics indicate that the higher the class ratio (Table 1) the better the performance, which makes sense as discussed that the model may underfit the minority class.…”

Data Fusion Analysis and Synthesis Framework for Improving Disaster Situation Awareness

“…Nowadays, UAVs are used in many application domains, such as: precision agriculture [28], construction and infrastructure inspection [29,30], and rescue of people [31] and also in the same context of this article-disaster management. For example, a literature review on the application of drones in the disaster management context is presented in [26].…”