FAAD: an unsupervised fast and accurate anomaly detection method for a multi-dimensional sequence over data stream

Li, Bin; Wang, Yijie; Yang, Dongsheng; Li, Yongmou; Ma, Xingkong

doi:10.1631/fitee.1800038

Cited by 15 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result of the attempts to overcome such a limitation, some strategies have been developed. For instance, NN‐DVI (Liu et al, 2018) focuses on defining neighborhoods to cope with regional drifts, whereas FAAD (Li et al, 2019) performs feature subset selection and feature sampling.…”

Section: Discussionmentioning

confidence: 99%

“…Finally, there are unsupervised methods that do not aim only establishing a drift detector. In this case, their final objective is a different task, like anomaly detection, as in FAAD (Li et al, 2019), and novel class detection, as in NM-DDM (Mustafa et al, 2017).…”

Section: Methodsmentioning

confidence: 99%

“…The second method is the unsupervised Fast and Accurate Anomaly Detection (FAAD) framework (Li et al, 2019), which was proposed to tackle unsupervised sequence‐anomaly detection in the context of problems involving multi‐dimensional sequences in data streams prone to concept drift. This kind of application refers to analyzing the ordering relationship in sequences so as to detect anomalies.…”

Section: Batch‐based Methodsmentioning

confidence: 99%

See 2 more Smart Citations

An overview of unsupervised drift detection methods

Gemaque

Costa

Giusti

et al. 2020

WIREs Data Min & Knowl

View full text Add to dashboard Cite

Practical applications involving big data, such as weather monitoring, identification of customer preferences, Internet log analysis, and sensors warnings require challenging data analysis, since these are examples of problems whose data are generated in streams and usually demand real-time analytics. Patterns in such data stream problems may change quickly. Consequently, machine learning models that operate in this context must be updated over time. This phenomenon is called concept drift in machine learning and data mining literature. Several different directions have been pursued to learn from data stream and to deal with concept drift. However, most drift detection methods consider that an instance's class label is available right after its prediction, since these methods work by monitoring the prediction results of a base classifier or an ensemble of classifiers. Nevertheless, this constraint is unrealistic in several practical problems. To cope with this constraint, some works are focused on proposing efficient unsupervised or semi-supervised concept drift detectors. While interesting and recent overview papers dedicated to supervised drift detectors have been published, the scenario is not the same in terms of unsupervised methods. Therefore, this work presents a comprehensive overview of approaches that tackle concept drift in classification problems in an unsupervised manner. Additional contribution includes a proposed taxonomy of state-of-the-art approaches for concept drift detection based on unsupervised strategies.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Batch‐based Methodsmentioning

confidence: 99%

See 1 more Smart Citation

An overview of unsupervised drift detection methods

Gemaque

Costa

Giusti

et al. 2020

WIREs Data Min & Knowl

View full text Add to dashboard Cite

show abstract

“…Another alternative is to study and analyze how the variation of a representative data group changes over time. Examples of this strategy can be found in [143,51,144,118].…”

Section: Data Distribution-based Approachmentioning

confidence: 99%

Continual Learning for Predictive Maintenance: Overview and Challenges

Hurtado¹,

Dario²,

Rudy³

et al. 2023

Preprint

View full text Add to dashboard Cite

A short review of data-driven solutions for Predictive Maintenance, non-stationary environments, and Continual Learning.• An overview and motivation of the intersection between Predictive Maintenance and Continual Learning.• The current challenge and trends of applying Continual Learning in current Predictive Maintenance benchmarks.• Proposal of a novel construction of benchmarks for Continual Learning, aiming to close the gap between lab and realistic environments.

show abstract

“…Some methods [ 9 , 22 ] gradually update statistical model without explicit detection of concept drift. Other studies attempt to detect concept drift in batch-based methods [ 23 , 24 , 25 , 26 ] and online methods [ 27 , 28 , 29 ]. While [ 30 , 31 ] rely on process control, detects concept drift using a multiple-window-based method like [ 32 ].…”

Section: Related Workmentioning

confidence: 99%

FlexSketch: Estimation of Probability Density for Stationary and Non-Stationary Data Streams

Park

Kim

2021

Sensors

View full text Add to dashboard Cite

Efficient and accurate estimation of the probability distribution of a data stream is an important problem in many sensor systems. It is especially challenging when the data stream is non-stationary, i.e., its probability distribution changes over time. Statistical models for non-stationary data streams demand agile adaptation for concept drift while tolerating temporal fluctuations. To this end, a statistical model needs to forget old data samples and to detect concept drift swiftly. In this paper, we propose FlexSketch, an online probability density estimation algorithm for data streams. Our algorithm uses an ensemble of histograms, each of which represents a different length of data history. FlexSketch updates each histogram for a new data sample and generates probability distribution by combining the ensemble of histograms while monitoring discrepancy between recent data and existing models periodically . When it detects concept drift, a new histogram is added to the ensemble and the oldest histogram is removed. This allows us to estimate the probability density function with high update speed and high accuracy using only limited memory. Experimental results demonstrate that our algorithm shows improved speed and accuracy compared to existing methods for both stationary and non-stationary data streams.

show abstract

FAAD: an unsupervised fast and accurate anomaly detection method for a multi-dimensional sequence over data stream

Cited by 15 publications

References 24 publications

An overview of unsupervised drift detection methods

An overview of unsupervised drift detection methods

Continual Learning for Predictive Maintenance: Overview and Challenges

FlexSketch: Estimation of Probability Density for Stationary and Non-Stationary Data Streams

Contact Info

Product

Resources

About