A multi-branch deep neural network model for failure prognostics based on multimodal data

Yang, Zhe; Baraldi, Piero; Zio, Enrico

doi:10.1016/j.jmsy.2021.01.007

Cited by 44 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, while the CNN's non-linear nature is one of the key ingredients to its flexibility for learning complex relationships within the data among other image classification approaches it makes also sensitive to initial conditions. A high variance/ low bias remedy to this problem has emerged from the "Ensembled Learning [32] which although typically utilizes data from a single source, recently has been applied to multimodal scenarios [33,34]. Herein findings on the improvement of the classification accuracy when combining the predictions from multiple neural networks (NN), seems to add the required bias that in turn counters the variance of a single trained NN.…”

Section: Model Selection and Architecturementioning

confidence: 99%

Online Quality Inspection Approach for Submerged Arc Welding (SAW) by Utilizing IR-RGB Multimodal Monitoring and Deep Learning

Stavropoulos

Papacharalampopoulos

Sabatakakis

2022

Lecture Notes in Mechanical Engineering

View full text Add to dashboard Cite

Online, Image-based monitoring of arc welding requires direct visual contact with the seam or the melt pool. During SAW, these regions are covered with flux, making it difficult to correlate temperature and spatial related features with the weld quality. In this study, by using a dual-camera setup, IR and RGB images depicting the irradiated flux during fillet welding of S335 structural steel beams are captured and utilized to develop a Deep Learning model capable of assessing the quality of the seam, according to four classes namely “no weld”, “good weld”, “porosity” and “undercut/overlap”, as they’ve emerged from visual offline inspection. The results proved that the camera-based monitoring could be a feasible online solution for defect classification in SAW with exceptional performance especially when a dual-modality setup is utilized. However, they’ve also pointed out that such a monitoring setup does not grand any real-world advantage when it comes to the classification of relatively large, defective seam regions.

show abstract

Section: Model Selection and Architecturementioning

confidence: 99%

Online Quality Inspection Approach for Submerged Arc Welding (SAW) by Utilizing IR-RGB Multimodal Monitoring and Deep Learning

Stavropoulos

Papacharalampopoulos

Sabatakakis

2022

Lecture Notes in Mechanical Engineering

View full text Add to dashboard Cite

show abstract

“…Many prognostic models have been developed in the literature, most of which focus on using time series-based degradation data Meeker, 2010, 2013;Shu et al, 2015;Liu et al, 2013;Gebraeel et al, 2005). Recently, prognostic models with imaging-based degradation data have been investigated and attracted more and more attention (Fang et al, 2019;Aydemir and Paynabar, 2019;Yang et al, 2021;Dong et al, 2021;Tang et al, 2021;Jiang et al, 2022). This is because comparing with time-series data, imaging data usually contains much richer information of the object being monitored, and imaging sensing technologies are noncontact and thus they can usually be easily deployed.…”

Section: Introductionmentioning

confidence: 99%

“…The existing imaging-based prognostic methods include deep learning-based models and statistical learning methods. Examples of the deep learning-based models designed for TTF prediction using imaging data include the ones developed by Aydemir and Paynabar (2019); Yang et al (2021); Dong et al (2021), and Jiang et al (2022). Although these models have worked relatively well, they usually provide point estimations of failure times, and it is challenging for them to quantify the uncertainty of predicted TTFs.…”

Section: Introductionmentioning

confidence: 99%

A Supervised Tensor Dimension Reduction-Based Prognostics Model for Applications with Incomplete Imaging Data

Zhou¹,

Fang²

2022

Preprint

View full text Add to dashboard Cite

Imaging data-based prognostic models focus on using an asset's degradation images to predict its time-to-failure (TTF). Most image-based prognostic models have two common limitations. First, they require degradation images to be complete (i.e., images are observed continuously and regularly over time). Second, they usually employ an unsupervised dimension reduction method to extract low-dimensional features, and then use the features for TTF prediction. Since unsupervised dimension reduction is conducted on the degradation images without the involvement of TTFs, there is no guarantee that the extracted features are effective for failure time prediction. To address these challenges, this article develops a supervised tensor dimension reduction-based prognostic model. The model first proposes a supervised dimension reduction method for tensor data. It uses historical TTFs to guide the detection of a tensor subspace to extract low-dimensional features from high-dimensional incomplete degradation imaging data. Next, the extracted features are used to construct a prognostic model based on (log)-location-scale regression. An optimization algorithm is proposed for parameter estimation, and closed-form solutions are derived under certain distributions. Simulated data and a real-world data set are used to validate the performance of the proposed model.

show abstract

“…36 Concatenated features were obtained from multimodal data through multi-branched deep neural network and higher performance was achieved for the degradation prediction of steam generators. 37 The feature level fusion aided by wavelet transform provided remarkable results with ANN as classifier for fault classification. 38 Features from vibration and force signals were fused using kernel principal component analysis (k-PCA) and support vector regression was applied for the prediction of tool wear.…”

Section: Introductionmentioning

confidence: 99%

A sensor fusion based approach for bearing fault diagnosis of rotating machine

Mian

Choudhary

Fatima

2021

Proceedings of the Institution of Mechanical Engineers, Part O:

View full text Add to dashboard Cite

Fault diagnosis in rotating machines plays a vital role in various industries. Bearing is the essential element of rotating machines, and early fault detection can reduce the maintenance cost and enhance machine availability. In complex industrial machinery, a single sensor has a limitation to capture complete information about fault conditions. Hence, there is a need to involve multiple sensors to diagnose all possible fault conditions effectively. In such situations, an efficient fusion of information is required to develop a reliable fault diagnosis system. In this work, a feature fusion approach is implemented using two different sensors, that is, a contact type vibration sensor and a non-invasive thermal imaging camera. Hilbert transform is applied to decompose raw vibration and thermal image data, and subsequently, features are extracted and fused into a single feature vector. However, the features are fused in a concatenation manner, but this stage has high dimensionality. Neighborhood component analysis (NCA) is applied to reduce this high dimensionality of the feature vector, followed by a relief algorithm (RA) to compute the relevance level to find the optimal features. Finally, these optimal features are used as an input feature vector to the support vector machine (SVM) to classify the faults. The proposed approach resulted in considerably improved classification accuracy and detection quality than individual sensors. Also, the relevance of the proposed approach is proved by comparing its performance with other prevalent feature fusion techniques.

show abstract

A multi-branch deep neural network model for failure prognostics based on multimodal data

Cited by 44 publications

References 31 publications

Online Quality Inspection Approach for Submerged Arc Welding (SAW) by Utilizing IR-RGB Multimodal Monitoring and Deep Learning

Online Quality Inspection Approach for Submerged Arc Welding (SAW) by Utilizing IR-RGB Multimodal Monitoring and Deep Learning

A Supervised Tensor Dimension Reduction-Based Prognostics Model for Applications with Incomplete Imaging Data

A sensor fusion based approach for bearing fault diagnosis of rotating machine

Contact Info

Product

Resources

About