Hybrid multimodal fusion with deep learning for rolling bearing fault diagnosis

Che, Changchang; Wang, Huawei; Ni, Xiaomei; Lin, Ruiguan

doi:10.1016/j.measurement.2020.108655

Cited by 48 publications

(25 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the video data, GaussianBlur function is called to denoise the video data, and then short time series video is generated by resampling, so as to improve the proportion of effective information. For facial video data, a video with a duration of 30 s is used as a segment; for gait video data, a video with a duration of 8 s is taken as a segment [ 27 , 28 ]. The preprocessed face and gait are F and G , respectively, so the video data can be expressed as

where t is determined by the size of video frames and f t and g t are single-frame facial and gait images.…”

Section: Simulation Experimentsmentioning

confidence: 99%

Using Convolutional Neural Networks for the Assessment Research of Mental Health

Liu

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

Existing mental health assessment methods mainly rely on experts’ experience, which has subjective bias, so convolutional neural networks are applied to mental health assessment to achieve the fusion of face, voice, and gait. Among them, the OpenPose algorithm is used to extract facial and posture features; openSMILE is used to extract voice features; and attention mechanism is introduced to reasonably allocate the weight values of different modal features. As can be seen, the effective identification and evaluation of 10 indicators such as mental health somatization, depression, and anxiety are realized. Simulation results show that the proposed method can accurately assess mental health. Here, the overall recognition accuracy can reach 77.20%, and the F1 value can reach 0.77. Compared with the recognition methods based on face single-mode fusion, face + voice dual-mode fusion, and face + voice + gait multimodal fusion, the recognition accuracy and F1 value of proposed method are improved to varying degrees, and the recognition effect is better, which has certain practical application value.

show abstract

where t is determined by the size of video frames and f t and g t are single-frame facial and gait images.…”

Section: Simulation Experimentsmentioning

confidence: 99%

Using Convolutional Neural Networks for the Assessment Research of Mental Health

Liu

2022

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

show abstract

“…Tang et al [17] decomposed raw vibration signals into intrinsic mode functions (IMFs) by CEEMD, and converted appropriate IMFs into 2-D images as the input of CNN for low-speed structural fault diagnosis. For various deep neural networks, the neurons in DAE and DBN models generally adopt full connection, while CNN network adopts local connection and weight sharing, which can greatly reduce the training of network parameters and improve the efficiency [18]- [20], in addition, VAE inherits the advantages of auto-encoder in unsupervised learning, while the characteristics of hidden layer features can better represent the input data than autoencoder [21]. In order to improve the safety of industrial process, some other methods such as transfer learning [22], hybrid method of fault detection and diagnosis [23] are also proposed to reduce risks.…”

Section: Introductionmentioning

confidence: 99%

Machinery New Emerge Fault Diagnosis Based on Deep Convolution Variational Autoencoder and Adaptive Label Propagation

She

Wang

2022

IEEE Access

View full text Add to dashboard Cite

In the research field of mechanical equipment fault diagnosis, usually only the existing fault types are identified, and the new emerge class of the fault is ignored, however, the new emerge fault class may also occur actually. In order to solve the problem, a novel fault diagnosis model based on deep convolution variational autoencoder network and adaptive label propagation (DCVAN-ALP) is proposed. Firstly, the initial high dimensional features are constructed by using the double tree complex wavelet packet method as the input of the model. Secondly, the convolutional neural network architecture is applied to construct the variational autoencoder, and the local and non-local characteristics of samples are embedded into the loss function for training, which is considered to improve the identification of hidden layer features of the neural network. Finally, t-SNE and the improved label propagation algorithm are adopted to process the hidden features of the neural network, which can achieve the purpose of diagnosing the existing fault class and especially new emerge fault class. Experimental results show that the proposed model can effectively extract the fault characteristics of the vibration signal, and it also has a significantly higher recognition accuracy rate than other typical deep learning methods and traditional classifiers in diagnosing new emerge fault class.

show abstract

“…Although transfer learning has achieved promising results in fault diagnosis of machinery, the methods commonly have the following shortcomings: first, most of these them still need a certain amount of labeled data, for example, reference [10] and [11] require more than 10 target training samples to achieve effective recognition accuracy; Second, we need to do a lot of preprocessing work, such as to extract features [12] from spectrum data rather than the original vibration data; and finally, these methods only transfer the simulation experiment data set to another simulation experiment data set [13], and the speed, loading, and fault degree of these data sets changed slightly, so the generalization ability of these methods are limited. To deal with the above-mentioned limitations, a new deep transfer learning network, named the transferred discriminator network (TD), is proposed for fault diagnose of rolling bearings in this paper.…”

Section: Introductionmentioning

confidence: 99%

A Fault Diagnosis Method for Rolling Bearings Based on Parameter Transfer Learning under Imbalance Data Sets

et al. 2021

View full text Add to dashboard Cite

Fault diagnosis under the condition of data sets or samples with only a few fault labels has become a hot spot in the field of machinery fault diagnosis. To solve this problem, a fault diagnosis method based on deep transfer learning is proposed. Firstly, the discriminator of the generative adversarial network (GAN) is improved by enhancing its sparsity, and then adopts the adversarial mechanism to continuously optimize the recognition ability of the discriminator; finally, the parameter transfer learning (PTL) method is applied to transfer the trained discriminator to target domain to solve the fault diagnosis problem with only a small number of label samples. Experimental results show that this method has good fault diagnosis performance.

show abstract

Hybrid multimodal fusion with deep learning for rolling bearing fault diagnosis

Cited by 48 publications

References 26 publications

Using Convolutional Neural Networks for the Assessment Research of Mental Health

Using Convolutional Neural Networks for the Assessment Research of Mental Health

Machinery New Emerge Fault Diagnosis Based on Deep Convolution Variational Autoencoder and Adaptive Label Propagation

A Fault Diagnosis Method for Rolling Bearings Based on Parameter Transfer Learning under Imbalance Data Sets

Contact Info

Product

Resources

About