Unsupervised outlier detection for time series by entropy and dynamic time warping

Benkabou, Seif-Eddine; Benabdeslem, Khalid; Canitia, Bruno

doi:10.1007/s10115-017-1067-8

Cited by 53 publications

(12 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [4,5], it was well organized in terms of practical application. Examples include credit card fraud detection [6,7], intrusion detection [8], defect detection [9], sensor data defect detection [10], time series data anomaly detection [11,12], detection of the abnormal data in terms of energy consumption [13], data quality improvement and cleaning [14,15], detection of the abnormal characters [16], and big data analysis [17,18].…”

Section: Related Workmentioning

confidence: 99%

A Study on the Preprocessing Method for Power System Applications Based on Polynomial and Standard Patterns

2022

View full text Add to dashboard Cite

Data-based decisions have been being made in various fields due to the development of sensors throughout the industries. Likewise, in the power system field, data-based decisions are being made in various tasks, including establishing distribution investment plans. However, in order for it to have validity, it is necessary to get rid of abnormal data or data with low representativeness of a temporary nature. Although in general, such a series of processes are done by preprocessing, the those of power system data should be handled not only noise but also data fluctuations caused by temporary change in operations such as load transfers, as mentioned above. In addition, the characteristics of load data of distribution lines (DLs) can be different depending on the characteristics of the load itself, the characteristics of the connected DLs, and regional characteristics of each DLs, so it is essential to propose and apply the optimized preprocessing method for each DL. In this study, therefore, an optimal preprocessing algorithm for each DL was proposed by mixing standard pattern calculations and polynomials based statistical method, and its appropriateness was verified by comparing the results with actual load transfer records. As a result of the verification, it was confirmed that the load transfer detection accuracy of the proposed method was 88.89%, and the maximum load of the target DL can be reduced up to 11.59% by removing the load transfer data.

show abstract

Section: Related Workmentioning

confidence: 99%

A Study on the Preprocessing Method for Power System Applications Based on Polynomial and Standard Patterns

2022

View full text Add to dashboard Cite

show abstract

“…Distance-based methods set thresholds regarding how far an instance deviates from its neighbours. The measurement can be defined distances, such as in k -nearest neighbours [20], or some kind of cost of separation such as decision tree-based methods [21]. Reconstructionbased methods catch patterns and calculate the expected values of instances to get the difference, i.e., residuals, and then use residuals to conduct labelling [22,23].…”

Section: Background and Related Workmentioning

confidence: 99%

A Robust Data‐Driven Method for Multiseasonality and Heteroscedasticity in Time Series Preprocessing

Sun

Shen

et al. 2021

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Internet of Things (IoT) is emerging, and 5G enables much more data transport from mobile and wireless sources. The data to be transmitted is too much compared to link capacity. Labelling data and transmit only useful part of the collected data or their features is a promising solution for this challenge. Abnormal data are valuable due to the need to train models and to detect anomalies when being compared to already overflowing normal data. Labelling can be done in data sources or edges to balance the load and computing between sources, edges, and centres. However, unsupervised labelling method is still a challenge preventing to implement the above solutions. Two main problems in unsupervised labelling are long-term dynamic multiseasonality and heteroscedasticity. This paper proposes a data-driven method to handle modelling and heteroscedasticity problems. The method contains the following main steps. First, raw data are preprocessed and grouped. Second, main models are built for each group. Third, models are adapted back to the original measured data to get raw residuals. Fourth, raw residuals go through deheteroscedasticity and become normalized residuals. Finally, normalized residuals are used to conduct anomaly detection. The experimental results with real-world data show that our method successfully increases receiver-operating characteristic (AUC) by about 30%.

show abstract

“…Measuring the regularity of dynamical systems is one of the hot topics in science and engineering. For example, it is used to investigate the health state in medical science [1,2], for real-time anomaly detection in dynamical networks [3], and for earthquake prediction [4]. Different statistical and mathematical methods are introduced to measure the degree of complexity in time series data, including the Kolmogorov complexity measure [5], the C 1 /C 2 complexity measure [5], and entropy [6].…”

Section: Introductionmentioning

confidence: 99%

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks

Velichko

Heidari

2021

Entropy

View full text Add to dashboard Cite

Measuring the predictability and complexity of time series using entropy is essential tool designing and controlling a nonlinear system. However, the existing methods have some drawbacks related to the strong dependence of entropy on the parameters of the methods. To overcome these difficulties, this study proposes a new method for estimating the entropy of a time series using the LogNNet neural network model. The LogNNet reservoir matrix is filled with time series elements according to our algorithm. The accuracy of the classification of images from the MNIST-10 database is considered as the entropy measure and denoted by NNetEn. The novelty of entropy calculation is that the time series is involved in mixing the input information in the reservoir. Greater complexity in the time series leads to a higher classification accuracy and higher NNetEn values. We introduce a new time series characteristic called time series learning inertia that determines the learning rate of the neural network. The robustness and efficiency of the method is verified on chaotic, periodic, random, binary, and constant time series. The comparison of NNetEn with other methods of entropy estimation demonstrates that our method is more robust and accurate and can be widely used in practice.

show abstract

Unsupervised outlier detection for time series by entropy and dynamic time warping

Cited by 53 publications

References 34 publications

A Study on the Preprocessing Method for Power System Applications Based on Polynomial and Standard Patterns

A Study on the Preprocessing Method for Power System Applications Based on Polynomial and Standard Patterns

A Robust Data‐Driven Method for Multiseasonality and Heteroscedasticity in Time Series Preprocessing

A Method for Estimating the Entropy of Time Series Using Artificial Neural Networks

Contact Info

Product

Resources

About