CloudDet: Interactive Visual Analysis of Anomalous Performances in Cloud Computing Systems

Abstract

Fig. 1. CloudDet facilitates the exploration of anomalous cloud computing performances through three levels of analysis: (a) anomaly ranking, (b) anomaly inspection, and (c) anomaly clustering. The figure showcases some exploration results with Bitbrains Datacenter traces data. Node (b1) contains both short and long term spikes, with no pattern in their occurrence times. Node (b2) shows a 12-hour periodic pattern for the performance metrics by observing the calendar chart in (a2), but encounters a spike in the process. Node (b3) shows many short-term and near-periodic spikes at the beginning and an abnormal long-term spike near the end. After collapsing the long-term one into a visual aggregation glyph in (b5), (b3) is updated and the latter temporal data "pop out", which shows a similar pattern as the beginning. Node (b4) shows a general periodic trend which is not apparent in (a4) by using the PCA analysis in (b6). Most of the nodes are clustered into three groups in (c), with each group displaying a rare but similar performance.Abstract-Detecting and analyzing potential anomalous performances in cloud computing systems is essential for avoiding losses to customers and ensuring the efficient operation of the systems. To this end, a variety of automated techniques have been developed to identify anomalies in cloud computing performance. These techniques are usually adopted to track the performance metrics of the system (e.g., CPU, memory, and disk I/O), represented by a multivariate time series. However, given the complex characteristics of cloud computing data, the effectiveness of these automated methods is affected. Thus, substantial human judgment on the automated analysis results is required for anomaly interpretation. In this paper, we present a unified visual analytics system named CloudDet to interactively detect, inspect, and diagnose anomalies in cloud computing systems. A novel unsupervised anomaly detection algorithm is developed to identify anomalies based on the specific temporal patterns of the given metrics data (e.g., the periodic pattern), the results of which are visualized in our system to indicate the occurrences of anomalies. Rich visualization and interaction designs are used to help understand the anomalies in the spatial and temporal context. We demonstrate the effectiveness of CloudDet through a quantitative evaluation, two case studies with real-world data, and interviews with domain experts.

A Systematic Review on Anomaly Detection for Cloud Computing Environments

“…References Time series analysis [162], [161], [76], [198], [83], [71], [210], [70] Bayesian learning [118], [85], [14], [103], [159], [50], [35], [128], [105], [157] Principal component analysis [195], [5], [18], [113], [108] Regression analysis [205], [54], [25], [89], [156] Logistic regression [131], [115], [26], [207] Hidden Markov model [79], [78], [11], [170] Markov chain [167], [52], [41] Gaussian mixture models [106], [13] Statistical tests [173], [114] Restricted Boltzmann machine [110], [ [201], [200], [63], Wavelet transform…”

Visual Progression Analysis of Event Sequence Data

“…Analysts can select the anomalies of interest to inspect their structural patterns in a node-link diagram and their invoked functions in a stacked timeline. [120] provides interactive visualization capabilities that enables analysts to inspect profile data and identify anomalous performances in cloud computing systems. This system combines multiple visualization modes such as glyph design and stacked line charts, to comprehensively monitor the performance of cloud computing systems from different aspects ( Fig.…”

MaraVis: Representation and Coordinated Intervention of Medical Encounters in Urban Marathon

“…In this paper, we first leverage the real-time streaming data and then adapt a distance-based algorithm to determine the potential outliers. Visualization techniques have been applied to support anomaly detection and facilitate decision-making [20,22,25,48]. Dimensionality reduction techniques are also applied to understand how data distribute in a multi-dimensional space, such as MDS [21], PCA [41] and t-SNE [29].…”

“…density, which is defined as: The parameter k can be arbitrarily determined based on users' experience regarding a given dataset [48]. Outliers with the LOF score larger than 1 indicate an isolated instance.…”