Quickest hub discovery in correlation graphs

Banerjee, Taposh; Hero, Alfred O.

doi:10.1109/acssc.2016.7869573

Cited by 4 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this univariate method is not suitable for detecting changes in the correlation between data-streams. A sequential test for detecting changes in the correlation between variables, as well as localizing the highly correlated variables in high-dimensional datastreams was proposed in [25]. This is a parametric method based on the assumption that the observed vectors are multivariate Gaussian distributed.…”

Section: Related Workmentioning

confidence: 99%

Self-Supervised Learning for Online Anomaly Detection in High-Dimensional Data Streams

2023

View full text Add to dashboard Cite

In this paper, we address the problem of detecting and learning anomalies in high-dimensional data-streams in real-time. Following a data-driven approach, we propose an online and multivariate anomaly detection method that is suitable for the timely and accurate detection of anomalies. We propose our method for both semi-supervised and supervised settings. By combining the semi-supervised and supervised algorithms, we present a self-supervised online learning algorithm in which the semi-supervised algorithm trains the supervised algorithm to improve its detection performance over time. The methods are comprehensively analyzed in terms of computational complexity, asymptotic optimality, and false alarm rate. The performances of the proposed algorithms are also evaluated using real-world cybersecurity datasets, that show a significant improvement over the state-of-the-art results.

show abstract

Section: Related Workmentioning

confidence: 99%

Self-Supervised Learning for Online Anomaly Detection in High-Dimensional Data Streams

2023

View full text Add to dashboard Cite

show abstract

“…However, this univariate method is not suitable for detecting changes in the correlation between data streams. A sequential test for detecting changes in the correlation between variables, as well as localizing the highly correlated variables, in high-dimensional data streams has been proposed in [ 23 ]. This is a parametric method based on the assumption that the observed vectors are multivariate Gaussian distributed.…”

Section: Related Workmentioning

confidence: 99%

Online Multivariate Anomaly Detection and Localization for High-Dimensional Settings

Mozaffari

Doshi

Yılmaz

2022

Sensors

View full text Add to dashboard Cite

This paper considers the real-time detection of abrupt and persistent anomalies in high-dimensional data streams. The goal is to detect anomalies quickly and accurately so that the appropriate countermeasures could be taken in time before the system possibly gets harmed. We propose a sequential and multivariate anomaly detection method that scales well to high-dimensional datasets. The proposed method follows a nonparametric, i.e., data-driven, and semi-supervised approach, i.e., trains only on nominal data. Thus, it is applicable to a wide range of applications and data types. Thanks to its multivariate nature, it can quickly and accurately detect challenging anomalies, such as changes in the correlation structure. Its asymptotic optimality and computational complexity are comprehensively analyzed. In conjunction with the detection method, an effective technique for localizing the anomalous data dimensions is also proposed. The practical use of proposed algorithms are demonstrated using synthetic and real data, and in variety of applications including seizure detection, DDoS attack detection, and video surveillance.

show abstract

“…However, this univariate method is not suitable for detecting changes in the correlation between data streams. A sequential test for detecting changes in the correlation between variables, as well as localizing the highly correlated variables, in highdimensional data streams has been proposed in [14]. This is a parametric method based on the assumption that the observed vectors are multivariate Gaussian distributed.…”

Section: A Related Workmentioning

confidence: 99%

Online Multivariate Anomaly Detection and Localization for High-dimensional Settings

Mozaffari¹,

Yılmaz²

2019

Preprint

View full text Add to dashboard Cite

This paper considers the real-time detection of anomalies in high-dimensional systems. The goal is to detect anomalies quickly and accurately so that the appropriate countermeasures could be taken in time, before the system possibly gets harmed. We propose a sequential and multivariate anomaly detection method that scales well to high-dimensional datasets. The proposed method follows a nonparametric, i.e., data-driven, and semi-supervised approach, i.e., trains only on nominal data. Thus, it is applicable to a wide range of applications and data types. Thanks to its multivariate nature, it can quickly and accurately detect challenging anomalies, such as changes in the correlation structure and stealth low-rate cyberattacks. Its asymptotic optimality and computational complexity are comprehensively analyzed. In conjunction with the detection method, an effective technique for localizing the anomalous data dimensions is also proposed. We further extend the proposed detection and localization methods to a supervised setup where an additional anomaly dataset is available, and combine the proposed semi-supervised and supervised algorithms to obtain an online learning algorithm under the semi-supervised framework. The practical use of proposed algorithms are demonstrated in DDoS attack mitigation, and their performances are evaluated using a real IoT-botnet dataset and simulations.

show abstract

Quickest hub discovery in correlation graphs

Cited by 4 publications

References 17 publications

Self-Supervised Learning for Online Anomaly Detection in High-Dimensional Data Streams

Self-Supervised Learning for Online Anomaly Detection in High-Dimensional Data Streams

Online Multivariate Anomaly Detection and Localization for High-Dimensional Settings

Online Multivariate Anomaly Detection and Localization for High-dimensional Settings

Contact Info

Product

Resources

About