Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and Hilbert transform

Kanarachos, Stratis; Christopoulos, Stavros-Richard G.; Chroneos, Alexander; Fitzpatrick, Michael E.

doi:10.1016/j.eswa.2017.04.028

Cited by 92 publications

(48 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In time series data anomalies can be categorized into: outliers, unusual data points significantly dissimilar to the remaining points in the data set, and anomaly patterns, which group fractions of data together, which are different from the majority of normal data. To deal with these categories, various anomaly detection algorithms have been developed that are classified into five major groups: probabilistic, distancebased, reconstruction-based, domain-based, and informationtheoretic-based [15]. Different types of algorithms are more applicable in various scenarios of data analysis, which brings to front the idea of a generic framework capable of combining several analytic tools and approaches.…”

Section: Methodsmentioning

confidence: 99%

“…These complex systems share at least two characteristics: a lack of mathematical model that accurately describes the system behaviour and a large amount of monitoring data that can be both historical and real-time. Therefore, for performing effective anomaly detection, it is necessary to rely on databased approaches and on smart methods of condition monitoring that often use computational intelligence and machine learning techniques [15]. The latter is especially important when some constraints are present that cannot be satisfied by human intervention with regard to decision making speed in life threatening situations (e.g.…”

Section: Introductionmentioning

confidence: 99%

“…Computational Intelligence (CI) and Machine Learning (ML) techniques have been successfully applied to problems involving the automation of anomaly detection in the process of condition monitoring [15], [17]. In particular, many studies have shown that artificial neural networks (ANN) can be a very effective classification tool to predict various anomalous conditions, such as the corrosion of oil and gas pipelines, turbine failure, and the like [8], [21], [25].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Generic Application of Deep Learning Framework for Real-Time Engineering Data Analysis

Majdani

Petrovski

2018

2018 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

The need for computer-assisted real-time anomaly detection in engineering data used for condition monitoring is apparent in various applications, including the oil and gas, automotive industries and many other engineering domains. To reduce the reliance on domain-specific experts' knowledge, this paper proposes a deep learning framework that can assist in building a versatile anomaly detection tool needed for effective condition monitoring. The framework enables building a computational anomaly detection model using different types of neural networks and supervised learning. While building such a model, three types of ANN units were compared: a recurrent neural network, a long short-term memory network, and a gated recurrent unit. Each of these units has been evaluated on two benchmark public datasets. The experimental results of this comparative study revealed that the LSTM network unit that uses the sigmoid activation function, the Mean Absolute Error as the objective Loss function and the Adam optimizer as the output layer showed the best performance and attained the accuracy of over 77 % in detecting anomalous values in the datasets. Having determined the best performing combination of the neural network components, a computational anomaly detection model was built within the framework, which was successfully evaluated on real-life engineering datasets comprising the timeseries datasets from an offshore installation in North Sea and another dataset from the automotive industry, which enabled exploring the anomaly classification capability of the proposed framework. Index Terms-Long short-term memory networks, deep learning framework, anomaly detection and classification, time series analysis, engineering applications.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Generic Application of Deep Learning Framework for Real-Time Engineering Data Analysis

Majdani

Petrovski

2018

2018 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

show abstract

“…A new training dataset is obtained by combining the classified dataset with original data and from that maximum shortage details is also identified [18]. Based on future queries a new classifier will be developed and can be widely used in medical field by the data collected from the patient who consumes large time and occupies vast area [19]. The artificial intelligence based neural network technique provides a step to make a final decision and the information received from neurons which is an interconnected nerve cells in surrounding [20].In an epoch of big data, the anomaly detection should be well-organized to process the large volume of data at real time without any loss of dynamic packet flow.…”

Section: Introductionmentioning

confidence: 99%

An Efficient Anomaly Detection Based On Optimal Deep Belief Network in Big Data

Dahiya,

Srivastva

2019

IJEAT

View full text Add to dashboard Cite

Nowadays, the internet and network service user’s counts are increasing and the data generation speed also very high. Then again, we see greater security dangers on the internet, enterprise network, websites and the network. Anomaly has been known as one of the effective cyber threats over the internet which increasing exponentially and thus overcomes the commonly used approaches for anomaly detection and classification. Anomaly detection is used in big data analytics to recognize the unexpected behaviour. The most commonly used characteristics in network environment are size and dimensionality, which are big datasets and also impose problems in recognizing useful patterns, For example, to identify the network traffic anomalies from the large datasets. Due to the enormous increase of computer network based facilities it is a challenge to perform fast and efficient anomaly detection. The anomaly recognition in big data sets is more useful to discover fraud and abnormal action. Here, we mainly focus on the problems regarding anomaly detection, so we introduce a novel machine learning based anomaly detection technique. Machine learning approach is used to enhance the anomaly detection speed which is very much useful to detect the anomaly from the large datasets. We evaluate the proposed framework by performing experiments with larger data sets and compare to several existing techniques such as fuzzy, SVM (Support Vector Machine) and PSO (Particle swarm optimization). It has shown 98% percentage of accuracy and the false rate of 0.002 % on proposed classifier. The experimental results illuminate that better performance than existing anomaly detection techniques in big data environment.

show abstract

“…Wavelets were used to detect spikes in the wavelet details of responses of a framed structure subjected to strong earthquake excitation, indicating damage occurrence [29]. A combined wavelets, neural networks, and Hilbert transform, inspired by the deep learning paradigm, was presented to form a new signal-processing algorithm [30]. This was further studied to identify the fault condition of a roller bearing using three types of deep neural network models such as deep Boltzmann machines, deep belief networks and stacked auto-encoders [31].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning-Based Damage, Load and Support Identification for a Composite Pipeline by Extracting Modal Macro Strains from Dynamic Excitations

et al. 2018

View full text Add to dashboard Cite

Featured Application: The proposed novel approach presented in this paper offers a highly effective and reliable scheme for damage detection and safety assessment of pipeline systems and can be incorporated as a practical and highly reliable structural health monitoring approach for the safe operationty of existing and new pipeline systems.Abstract: Modal macro strain-based damage identification is a promising approach since it has the advantages of high sensitivity and effectiveness over other related methods. In this paper, a basalt fiber-reinforced polymer (BFRP) pipeline system is used for analysis by using long-gauge distributed fiber Bragg grating (FBG) sensors. Dynamic macro strain responses are extracted to form modal macro strain (MMS) vectors. Both longitudinal distribution and circumferential distribution plots of MMS are compared and analyzed. Results show these plots can reflect damage information of the pipeline based on the previous work carried out by the authors. However, these plots may not be good choices for accurate detection of damage information since the model is 3D and has different flexural and torsional effects. Therefore, by extracting MMS information in the circumferential distribution plots, a novel deep neural network is employed to train and test these images, which reflect the important and key information of modal variance in the pipe system. Results show that the proposed Deep Learning based approach is a promising way to inherently identify damage types, location of the excitation load and support locations, especially when the structural types are complicated and the ambient environment is changing.

show abstract

Detecting anomalies in time series data via a deep learning algorithm combining wavelets, neural networks and Hilbert transform

Cited by 92 publications

References 41 publications

Generic Application of Deep Learning Framework for Real-Time Engineering Data Analysis

Generic Application of Deep Learning Framework for Real-Time Engineering Data Analysis

An Efficient Anomaly Detection Based On Optimal Deep Belief Network in Big Data

Deep Learning-Based Damage, Load and Support Identification for a Composite Pipeline by Extracting Modal Macro Strains from Dynamic Excitations

Contact Info

Product

Resources

About