Distress Image Retrieval for Infrastructure Maintenance via Self-Trained Deep Metric Learning Using Experts’ Knowledge

Abstract

Distress image retrieval for infrastructure maintenance via self-trained deep metric learning using experts' knowledge is proposed in this paper. Since engineers take multiple images of a single distress part for inspection of road structures, it is necessary to construct a similar distress image retrieval method considering the input of multiple images to support determination of the level of deterioration. Thus, the construction of an image retrieval method while selecting an effective input from multiple images is described in this paper. The proposed method performs deep metric learning by using a small number of effective images labeled by experts' knowledge with information about their effectiveness and a large number of unlabeled images via a self-training approach. Specifically, an end-to-end learning approach that performs retraining of the model by assigning pseudo-labels to these unlabeled images according to the output confidence of the model is achieved. Thus, the proposed method can select an effective image from multiple images that are input at the retrieval as a query image. This is the main contribution of this paper. As a result, the proposed method realizes highly accurate retrieval of similar distress images considering the actual situation of inspection in which multiple images of a distress part are input. INDEX TERMS Distress image retrieval, self-trained approach, pseudo-label, deep metric learning.

Distress Detection in Subway Tunnel Images via Data Augmentation Based on Selective Image Cropping and Patching

“…Under these circumstances, advanced infrastructure maintenance techniques that can reduce costs and the burden on engineers are needed [10][11][12][13]. Some infrastructure maintenance methods have been proposed based on computer vision with images of infrastructure [14][15][16]. These methods can be roughly divided into the following three categories: image processing-based, machine learning-based, and deep learning-based methods.…”

Reliable Estimation of Deterioration Levels via Late Fusion Using Multi-View Distress Images for Practical Inspection

“…In order to accurately judge the deterioration levels, the engineers are required to check the surrounding conditions of the distress regions and whether or not the distress is progressive. For this purpose, the engineers take multi-view images of a single inspection point from different angles and distances, and judge the final deterioration levels based on these multi-view images [12]. Therefore, it is necessary to construct a model that can estimate the deterioration level from multi-view images.…”