Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing

Self Cite

In this study, we investigate a novel idea of using synthetic images of bolts which are generated from a graphical model to train a deep learning model for loosened bolt detection. Firstly, a framework for bolt-loosening detection using image-based deep learning and computer graphics is proposed. Next, the feasibility of the proposed framework is demonstrated through the bolt-loosening monitoring of a lab-scaled bolted joint model. For practicality, the proposed idea is evaluated on the real-scale bolted connections of a historical truss bridge in Danang, Vietnam. The results show that the deep learning model trained by the synthesized images can achieve accurate bolt recognitions and looseness detections. The proposed methodology could help to reduce the time and cost associated with the collection of high-quality training data and further accelerate the applicability of vision-based deep learning models trained on synthetic data in practice.

Section: Image-based Deep Learning Methods For Bolt-loosening Detectionmentioning

confidence: 99%

Bolt-Loosening Monitoring Framework Using an Image-Based Deep Learning and Graphical Model

Pham

Kim

et al. 2020

Self Cite

“…Kang and Cha [14,159] developed an autonomous UAV system for SHM while using ultrasonic beacons to replace the role of GPS that performs poorly in partially covered places, such as under bridge decks. In addition, Huynh et al [160] used a UAV for the quasi-real-time inspection of connection bolts on a full-scale girder bridge. Dorafshan et al [75] examined the performance of different UASs for detecting cracks in steel bridges and concluded that instability in GPS-denied and windy environment might pose major challenges for UAS-assisted inspections.…”

Section: Applications Of Uavs and Portable Smartphones For Dl-based Shmmentioning

confidence: 99%

Data-Driven Structural Health Monitoring and Damage Detection through Deep Learning: State-of-the-Art Review

Azimi

Eslamlou

Pekcan

2020

334

158

Data-driven methods in structural health monitoring (SHM) is gaining popularity due to recent technological advancements in sensors, as well as high-speed internet and cloud-based computation. Since the introduction of deep learning (DL) in civil engineering, particularly in SHM, this emerging and promising tool has attracted significant attention among researchers. The main goal of this paper is to review the latest publications in SHM using emerging DL-based methods and provide readers with an overall understanding of various SHM applications. After a brief introduction, an overview of various DL methods (e.g., deep neural networks, transfer learning, etc.) is presented. The procedure and application of vibration-based, vision-based monitoring, along with some of the recent technologies used for SHM, such as sensors, unmanned aerial vehicles (UAVs), etc. are discussed. The review concludes with prospects and potential limitations of DL-based methods in SHM applications.

“…Although many academic works have explored different methods of detecting bolt loosening, including vibration-based measurements [ 2 , 3 ], electro-mechanical impedance methods [ 4 , 5 , 6 , 7 , 8 ], electrical conductivity techniques [ 9 ], ultrasonic-based measurements [ 10 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 ], and vision-based methods [ 21 , 22 , 23 , 24 , 25 , 26 ], much less research has been done on the applications of machine learning (ML) or deep learning (DL) algorithms in this field. Recently, ML and DL have become the breakthrough tools, particularly in the field of computer vision, to overcome the limitations of conventional structural health monitoring (SHM) and non-destructive evaluation (NDE).…”

Section: Introductionmentioning

confidence: 99%

“…As introduced in a review study by Azimi et al [ 27 ], ML and DL algorithms can be successfully applied to a wide range of applications in SHM, such as cracks, corrosion detection and structural component recognition [ 28 , 29 ]. Recent studies utilizing vision-based techniques with ML and DL algorithms for bolt looseness detection have been conducted [ 21 , 24 , 25 , 30 ]. These studies can be summarized as follows.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Artificial Intelligence-Based Bolt Loosening Diagnosis Using Deep Learning Algorithms for Laser Ultrasonic Wave Propagation Data

Tran

Kim

Tola

et al. 2020

The application of deep learning (DL) algorithms to non-destructive evaluation (NDE) is now becoming one of the most attractive topics in this field. As a contribution to such research, this study aims to investigate the application of DL algorithms for detecting and estimating the looseness in bolted joints using a laser ultrasonic technique. This research was conducted based on a hypothesis regarding the relationship between the true contact area of the bolt head-plate and the guided wave energy lost while the ultrasonic waves pass through it. First, a Q-switched Nd:YAG pulsed laser and an acoustic emission sensor were used as exciting and sensing ultrasonic signals, respectively. Then, a 3D full-field ultrasonic data set was created using an ultrasonic wave propagation imaging (UWPI) process, after which several signal processing techniques were applied to generate the processed data. By using a deep convolutional neural network (DCNN) with a VGG-like architecture based regression model, the estimated error was calculated to compare the performance of a DCNN on different processed data set. The proposed approach was also compared with a K-nearest neighbor, support vector regression, and deep artificial neural network for regression to demonstrate its robustness. Consequently, it was found that the proposed approach shows potential for the incorporation of laser-generated ultrasound and DL algorithms. In addition, the signal processing technique has been shown to have an important impact on the DL performance for automatic looseness estimation.