Relational Synthesis of Text and Numeric Data for Anomaly Detection on Computing System Logs

Baseman, Elisabeth; Blanchard, Sean; Zongze, Li; Fu, Song

doi:10.1109/icmla.2016.0158

Cited by 17 publications

(15 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The algorithm thus does not detect these bursts as anomalies. (2) Intensive bursts cause CVAE and PCA to detect a relatively small burst as a normal. Figure 6 is a histogram of detected anomalies per log category.…”

Section: Results Overviewmentioning

confidence: 99%

“…Many studies have been conducted for finding anomalies and their root causes [2], [4], [5] in log data. Zhong et al [3] proposed an anomaly detection method for both device and network errors with fine log time series feature creation.…”

Section: Related Workmentioning

confidence: 99%

“…However, it is not easy for network operators to investigate the details of network problems with the logs because of their diverse and massive nature. Many log analysis methods for finding anomalies and their root causes have been proposed to overcome this problem [2]- [7]. In many cases, one first classifies the logs by their message type (i.e., log template) then treats them as statistical time series to be later processed through statistical analysis.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Latent Variable Based Anomaly Detection in Network System Logs

Otomo

Kobayashi

Fukuda

et al. 2019

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

System logs are useful to understand the status of and detect faults in large scale networks. However, due to their diversity and volume of these logs, log analysis requires much time and effort. In this paper, we propose a log event anomaly detection method for large-scale networks without pre-processing and feature extraction. The key idea is to embed a large amount of diverse data into hidden states by using latent variables. We evaluate our method with 12 months of system logs obtained from a nationwide academic network in Japan. Through comparisons with Kleinberg's univariate burst detection and a traditional multivariate analysis (i.e., PCA), we demonstrate that our proposed method achieves 14.5% higher recall and 3% higher precision than PCA. A case study shows detected anomalies are effective information for troubleshooting of network system faults.

show abstract

Section: Results Overviewmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Latent Variable Based Anomaly Detection in Network System Logs

Otomo

Kobayashi

Fukuda

et al. 2019

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

show abstract

“…As a final step data mining techniques like clustering and association analysis are deployed to discover the interesting patterns and evaluate them to make final decisions. Efforts [77,95], used this approach to extract interesting patterns for the anomaly detection in NVE. Baseman et al [77], presented a method for anomaly detection based on syslog data collected from VMs.…”

Section: Log Analytics and Data Miningmentioning

confidence: 99%

“…Efforts [77,95], used this approach to extract interesting patterns for the anomaly detection in NVE. Baseman et al [77], presented a method for anomaly detection based on syslog data collected from VMs. They extract the Infomap clusters on textual data; and relational features on numeric data and combine their features with keyword counts to create a single data set.…”

Section: Log Analytics and Data Miningmentioning

confidence: 99%

A Survey of Fault Management in Network Virtualization Environments: Challenges and Solutions

Cherrared

Imadali

Fabre

et al. 2019

IEEE Trans. Netw. Serv. Manage.

View full text Add to dashboard Cite

The advent of 5G and the ever increasing stringent requirements in terms of bandwidth, latency, and quality of service pushes the boundaries of what is feasible with legacy Mobile Network Operators' technologies. Network Functions Virtualization (NFV) is a promising attempt at solving some of these challenges that is widely adopted by the industry and specified by the standardization bodies. In essence, NFV is about running Network Functions (NFs) as virtualized workloads on commodity hardware. This may optimize deployment costs and simplify the lifecycle management of NFs, but it introduces new fault management challenges and issues. In this paper, we propose a comprehensive state of the art of fault management techniques. We address the impact of virtualization in fault management. We propose a new classification of the recent fault management research achievements in the network virtualization environments and compare their major contributions and shortcomings.

show abstract