Log Pattern Mining for Distributed System Maintenance

Abstract

Due to the complexity of the network structure, log analysis is usually necessary for the maintenance of network-based distributed systems since logs record rich information about the system behaviors. In recent years, numerous works have been proposed for log analysis; however, they ignore temporal relationships between logs. In this paper, we target on the problem of mining informative patterns from temporal log data. We propose an approach to discover sequential patterns from event sequences with temporal regularities. Discovered patterns are useful for engineers to understand the behaviors of a network-based distributed system. To solve the well-known problem of pattern explosion, we resort to the minimum description length (MDL) principle and take a step forward in summarizing the temporal relationships between adjacent events of a pattern. Experiments on real log datasets prove the efficiency and effectiveness of our method.

Log File Analysis Based on Machine Learning: A Survey

“…Chen et al [30] studied the issue of extracting useful patterns using temporal log data. They present a new algorithm Discovering Patterns from Temporal Sequences (DTS) algorithm for extracting sequential patterns from temporally regular sequence data.…”

A Model for Trend Analysis in the Online Shopping Scenario Using Multilevel Hesitation Pattern Mining

“…e objective of analysis is to know the buying patterns of customers on the basis of their liking and disliking. As evident from the literature, the analytics act has been exercised to reveal various types of patterns such as Frequent Patterns [1][2][3][4][5], Profitable Patterns [6], Conditional Patterns [7], Calendar-Based Patterns [8], and Log Pattern Mining [9] using various techniques of pattern mining [10]. Moreover, after the success of mining knowledge from datasets, researchers deal with certain specific situations and perform various tasks such as mining on data streams [11,12], recognition of handwritten expression [13], investigating customer buying behavior through Visual Market Basket Analysis (VMBA) [14], automated assessment of shopping behavior [15,16], applying additional interestingness measures for association rule mining [17], and conditional discriminative pattern mining [18], and researchers also have to deal to improve the implementation of pattern mining algorithms using time stamp uncertainties and temporal constraints [19], privacy of frequent itemset mining using randomized response [20], and finding infrequent itemset to discover the negative association rule [21].…”