Granger Causality for Time-Series Anomaly Detection

Abstract

Abstract-Recent developments in industrial systems provide us with a large amount of time series data from sensors, logs, system settings and physical measurements, etc. These data are extremely valuable for providing insights about the complex systems and could be used to detect anomalies at early stages. However, the special characteristics of these time series data, such as high dimensions and complex dependencies between variables, as well as its massive volume, pose great challenges to existing anomaly detection algorithms. In this paper, we propose Granger graphical models as an effective and scalable approach for anomaly detection whose results can be readily interpreted. Specifically, Granger graphical models are a family of graphical models that exploit the temporal dependencies between variables by applying L 1-regularized learning to Granger causality. Our goal is to efficiently compute a robust "correlation anomaly" score for each variable via Granger graphical models that can provide insights on the possible reasons of anomalies. We evaluate the effectiveness of our proposed algorithms on both synthetic and application datasets. The results show the proposed algorithm achieves significantly better performance than other baseline algorithms and is scalable for large-scale applications.

Using causal discovery for feature selection in multivariate numerical time series

“…A number of approaches have been proposed for causal discovery in time series. Granger causality (Granger 1969) is a well-known method and some of its extensions (Arnold et al 2007;Lozano et al 2009;Chen et al 2004;Eichler 2012) have gained successes across many domains (especially in economics) due to their simplicity, robustness and extendibility (Brovelli et al 2004;Hiemstra and Jones 1994;Kim 2012;Qiu et al 2012).…”

Data/Feature Distributed Stochastic Coordinate Descent for Logistic Regression

“…Logistic regression is a widely-used method to solve classification problems in data mining. It has been applied to many applications such as life science [11], threat classification and temporal link analysis [7], anomaly detection [19], collaborative filtering [24] and text processing [17].…”

“…How to design the distributed algorithm efficiently? Logistic regression, or L 1 regularized loss minimization in general, is a crucial task with many applications including biological data mining [11], threat classification [7], text processing [9], matrix factorization [6,25], anomaly detection [19], etc. The major algorithms for learning the parameter for the logistic regression problem are descent based algorithms, including Stochastic Gradient Descent (SGD) and Stochastic Coordinate Descent (SCD).…”

Discovery of Causal Time Intervals

“…Following the work [2], [20] detects causality of spatial time series, [18] proposes to use hidden Markov Random Field method, [17] handles extreme values in time series, [4] detects Granger causality from irregular time series, and [5] presents Copula-Granger method to efficiently capture non-linearity in the data. Learning temporal causal graph has been applied to biology applications [25], climate analysis [9], microbiology [19], fMRI data analysis [24], anomaly detection [23], and longitudinal analysis [26]. However, none of these work address the issues that causality could change over time, and that Copyright © by SIAM Unauthorized reproduction of this article is prohibited 805 Downloaded 04/27/19 to 52.183.12.225.…”