Cloud Based Fault Diagnosis by Convolutional Neural Network as Time–Frequency RGB Image Recognition of Industrial Machine Vibration with Internet of Things Connectivity

Abstract

The human-centric and resilient European industry called Industry 5.0 requires a long lifetime of machines to reduce electronic waste. The appropriate way to handle this problem is to apply a diagnostic system capable of remotely detecting, isolating, and identifying faults. The authors present usage of HTTP/1.1 protocol for batch processing as a fault diagnosis server. Data are sent by microcontroller HTTP client in JSON format to the diagnosis server. Moreover, the MQTT protocol was used for stream (micro batch) processing from microcontroller client to two fault diagnosis clients. The first fault diagnosis MQTT client uses only frequency data for evaluation. The authors’ enhancement to standard fast Fourier transform (FFT) was their usage of sliding discrete Fourier transform (rSDFT, mSDFT, gSDFT, and oSDFT) which allows recursively updating the spectrum based on a new sample in the time domain and previous results in the frequency domain. This approach allows to reduce the computational cost. The second approach of the MQTT client for fault diagnosis uses short-time Fourier transform (STFT) to transform IMU 6 DOF sensor data into six spectrograms that are combined into an RGB image. All three-axis accelerometer and three-axis gyroscope data are used to obtain a time-frequency RGB image. The diagnosis of the machine is performed by a trained convolutional neural network suitable for RGB image recognition. Prediction result is returned as a JSON object with predicted state and probability of each state. For HTTP, the fault diagnosis result is sent in response, and for MQTT, it is send to prediction topic. Both protocols and both proposed approaches are suitable for fault diagnosis based on the mechanical vibration of the rotary machine and were tested in demonstration.

Transformer Faults Classification Based on Convolution Neural Network

“…The primary metric for comparing classifiers was the F1-score. F1-score, recall and Precision are computed as shown in the following equations 1, 2 and 3 [24]. Tables 5 and 6 show the parameters of the variance and the 2 nd harmonic sequential models respectively.…”

“…The main characteristic of CNN makes it better than standard ANNs for recognition [23]. CNN is an effective tool for recognizing multi-spectrograms that are structured into numerical data for diagnosing faults, eliminating the requirement of selecting the vibration axis beforehand [24]. The proposed ML method uses a CNN framework that performs discrimination between inrush and faulty currents.…”

Convolutional Neural Network with Attention Mechanism and Visual Vibration Signal Analysis for Bearing Fault Diagnosis

“…Additionally, InceptionResNetV2 was employed to accommodate the non-rigid characteristics and larger receptive field in the graph [ 21 ]. Łuczak proposed a fault diagnosis method using a CNN to process spectrograms for fault diagnosis [ 22 ]. In this method, short-time Fourier transform (STFT) is applied to transform 6 DOF sensor data into spectrograms that are combined into an RGB image.…”

Machine Fault Diagnosis through Vibration Analysis: Time Series Conversion to Grayscale and RGB Images for Recognition via Convolutional Neural Networks

“…The choice depends on the specific machine and the fault characteristics it aims to detect. Common sensors are divided by the type of measurement: (a) mechanical quantities like vibration (a popular choice due to its sensitivity to faults) [1][2][3][4][5], displacement [6], torque [7,8], and angular velocity/position [9,10]; (b) electrical quantities like current [11,12] and voltage [13,14], can reveal issues related to power delivery and motor health; and (c) other signals like temperature (inner/outer) [15,16], sound [17][18][19], and even chemical analysis [20,21] can be valuable for specific fault types. Beyond traditional sensors, recent research explores image-based diagnostics using cameras [22][23][24][25] and signals converted into virtual images [12,[26][27][28][29][30].…”