Partly interpretable transformer through binary arborescent filter for intelligent bearing fault diagnosis

Jiao, Zhiyuan; Pan, Liren; Wang, Fan; Xu, Zhenying; Chen, Chao

doi:10.1016/j.measurement.2022.111950

Cited by 16 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To overcome this limitation, the token encoder module is utilized to encode the input tokens by a matrix with hidden information 40 . It facilitates information exchange between tokens and mines the relationship between different features of plant diseases, resulting in an improved feature representation 41,42 . The token encoder module consists of two multi-label encoder blocks.…”

Section: Token Encoder Modulementioning

confidence: 99%

See 1 more Smart Citation

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method

Yang,

Li,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The growth of plants is threatened by numerous diseases. Accurate and timely identification of these diseases is crucial to prevent disease spreading. Many deep learning-based methods have been proposed for identifying leaf diseases. However, these methods often combine plant, leaf disease, and severity into one category or treat them separately, resulting in a large number of categories or complex network structures. Given this, this paper proposes a novel leaf disease identification network (LDI-NET) using a multi-label method. It is quite special because it can identify plant type, leaf disease and severity simultaneously using a single straightforward branch model without increasing the number of categories and avoiding extra branches. It consists of three modules, i.e., a feature tokenizer module, a token encoder module and a multi-label decoder module. The LDI-NET works as follows: Firstly, the feature tokenizer module is designed to enhance the capability of extracting local and long-range global contextual features by leveraging the strengths of convolutional neural networks and transformers. Secondly, the token encoder module is utilized to obtain context-rich tokens that can establish relationships among the plant, leaf disease and severity. Thirdly, the multi-label decoder module combined with a residual structure is utilized to fuse shallow and deep contextual features for better utilization of different-level features. This allows the identification of plant type, leaf disease, and severity simultaneously. Experiments show that the proposed LDI-NET outperforms the prevalent methods using the publicly available AI challenger 2018 dataset.

show abstract

Section: Token Encoder Modulementioning

confidence: 99%

“…Figure 3a illustrates the structure of the token encoder block, which consists of a multi-head self-attention (MSA) and an MLP. Additionally, necessary residual connections and norms are added 41 to prevent model degradation and accelerate convergence during training.…”

Section: Token Encoder Modulementioning

confidence: 99%

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method

Yang,

Li,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…The method based on class activation mapping (CAM) [7,8] is also an important method. The article [9] uses the Kurtogram to decompose the signal and a heat map to explain the model. Article [10] proposed the gradient score CAM (GS-CAM) method to make it more suitable for the proposed attention mechanism.…”

Section: Introductionmentioning

confidence: 99%

Interpretable parallel channel encoding convolutional neural network for bearing fault diagnosis

Tong,

Du,

Jiang

et al. 2024

Meas. Sci. Technol.

View full text Add to dashboard Cite

Interpretability plays a crucial role in the application of neural networks for fault diagnosis. Integrating preprocessing methods into neural networks can enhance interpretability while preserving their "end-to-end" characteristics. However, when only redesigning the first layer, subsequent structures still exhibit limited transparency. Additionally, traditional convolution structure is ill-suited for analyzing the readable feature maps derived from vibration signals. To address these challenges, this paper proposes a novel convolution structure for the parameterized signal processing function of the first-layer convolution kernel. This structure incorporates channel mixing for feature augmentation, designs a condensed feature encoder for aggregating and compressing features channel-by-channel, ensures the interpretability of feature map processing, and obtains condensed feature codes to propose SA-LRP (Smooth Activation Layer-wise Relevance Propagation) method that to perform interpretability analysis. Additionally, cubic feature screening is implemented for diagnostic classification to improve structural fitness. We design experiments using multiple datasets to test various indicators of the structure. The results confirm that connecting our convolution architecture for subsequent analysis outperforms other convolution architectures for the convolution kernel of the first-layer parameterized signal processing function. The interpretability of the model is evaluated through SA-LRP method and validates the interpretability of the model.

show abstract

“…Yang et al divided the original signal and finally constructed a diagnosis method using signal Transformer with pure attention mechanism [20]. Jiao et al interpreted the embedding module construction features as a classifier to define different fault types and then constructed a fault diagnosis method based on the binary arborescent filter Transformer for rotating machinery, achieving excellent classification results [21]. It is noteworthy that positional information in 1D vibration signal sequences often utilized in bearing diagnosis tasks is also extremely important.…”

Section: Introductionmentioning

confidence: 99%

Bearing fault diagnosis method using the joint feature extraction of Transformer and ResNet

Hou

Lian

Chu

2023

Meas. Sci. Technol.

View full text Add to dashboard Cite

The failure of rotating machinery can be prevented and eliminated by regular diagnosis of bearings. In the deep learning model of bearing fault diagnosis driven by big data, there often exist problems such as data acquisition difficulties, data distribution imbalance and high noise in samples. This study proposes a novel bearing fault diagnosis method using the joint feature extraction of Transformer and ResNet coupled with transfer learning strategy (TL-TAR) to overcome the abovementioned issues. First, the data is transmitted to the Transformer encoder and ResNet architecture respectively, where the input obtained by the encoder needs to separate features and word embedding through one-dimensional convolutional layer. Next, the feature sequences mined using encoder and ResNet are connected and classified. Moreover, the transfer learning strategy with model fine-tuning is exploited to reduce the train difficulty of the proposed method in new tasks. Experiments on two bearing fault datasets show that the proposed method can effectively combine the characteristics of both architectures, and the prediction accuracy outperform traditional deep learning networks in high noise environments.

show abstract

Partly interpretable transformer through binary arborescent filter for intelligent bearing fault diagnosis

Cited by 16 publications

References 24 publications

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method

A novel plant type, leaf disease and severity identification framework using CNN and transformer with multi-label method

Interpretable parallel channel encoding convolutional neural network for bearing fault diagnosis

Bearing fault diagnosis method using the joint feature extraction of Transformer and ResNet

Contact Info

Product

Resources

About