Enhancing Pneumonia Detection using Vision Transformer with Dynamic Mapping Re-Attention Mechanism

Abstract

Fault detection and diagnosis (FDD) is a crucial task for ensuring the safety and efficiency of industrial processes. We propose a novel FDD methodology for the Tennessee Eastman Process (TEP), a widely used benchmark for chemical process control. The model employs two separate Transformer branches, enabling independent processing of input data and potential extraction of diverse information. A novel attention mechanism, Gated Dynamic Learnable Attention (GDLAttention), is introduced which integrates a gating mechanism and dynamic learning capabilities. The gating mechanism modulates the attention weights, allowing the model to focus on the most relevant parts of the input. The dynamic learning approach adapts the attention strategy during training, potentially leading to improved performance. The attention mechanism uses a bilinear similarity function, providing greater flexibility in capturing complex relationships between query and key vectors. In order to assess the effectiveness of our approach, we tested it against 21 and 18 distinct fault scenarios in TEP, and compared its performance with several established FDD techniques. The outcomes indicate that the method outperforms others in terms of accuracy, false alarm rate, and misclassification rate. This underscores the robustness and efficacy of the approach for FDD in intricate industrial processes.

Tuning of Novel Fractional Order Fuzzy PID Controller for Automatic Voltage Regulator using Grasshopper Optimization Algorithm

“…Moreover, the attention mechanism in Transformers is computationally efficient as it allows for parallel computation across the sequence, unlike RNNs (Recurrent Neural Network) which require sequential computation (Samii et al, 2023). This makes Transformers faster and more scalable for large datasets (Labbaf Khaniki, Mirzaeibonehkhater and Manthouri, 2023). This research introduces a pioneering methodology for time series prediction of Bitcoin, Ethereum, and Litecoin.…”