Discovering lag intervals for temporal dependencies

Tang, Liang; Li, Tao; Shwartz, Larisa

doi:10.1145/2339530.2339633

Cited by 25 publications

(13 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spatio-temporal data sets, which include the location and time of events as crucial fields for analysis, are particularly appropriate for this research project because much analysis can be usefully be performed on them in a domain-independent manner. Time series data sets are well suited to performing multiple independent analyses, e.g., looking for patterns at different amounts of lag [41] and different levels of granularity, e.g., hour, day, week, month, etc. [42], and such analyses can be performed on an extremely wide variety of data sets.…”

Section: 32mentioning

confidence: 99%

What's the big deal about big data?

Cercone

2015

BDIA

View full text Add to dashboard Cite

This position paper is based on a major cooperative research and development proposal to form a Big Data Research, Analytics, and Information Network (BRAIN). Challenges presented by Big Data research are introduced and several projects are sketched in four important Big Data research theme areas, the solutions of which will further decision making in these areas of investigation. The four themes are large-scale data analytics and cloud computing, computational biology, health informatics, and interactive content analytics. These theme areas are certainly not inclusive, rather indicative of the wide variety to which Big Data now occupies decision analytics. The importance of training highly qualified personnel (HQP), knowledge mobilization and novelty are discussed.

show abstract

Section: 32mentioning

confidence: 99%

What's the big deal about big data?

Cercone

2015

BDIA

View full text Add to dashboard Cite

show abstract

“…See example, [13], [21], [16], [32]. Different types of patterns, such as (partially) periodic patterns, event bursts, and mutually dependent patterns were introduced to describe system management events.…”

Section: Related Workmentioning

confidence: 99%

An integrated framework for optimizing automatic monitoring systems in large IT infrastructures

Tang

Shwartz

et al. 2013

Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Self Cite

View full text Add to dashboard Cite

The competitive business climate and the complexity of IT environments dictate efficient and cost-effective service delivery and support of IT services. These are largely achieved by automating routine maintenance procedures, including problem detection, determination and resolution. System monitoring provides an effective and reliable means for problem detection. Coupled with automated ticket creation, it ensures that a degradation of the vital signs, defined by acceptable thresholds or monitoring conditions, is flagged as a problem candidate and sent to supporting personnel as an incident ticket. This paper describes an integrated framework for minimizing false positive tickets and maximizing the monitoring coverage for system faults.In particular, the integrated framework defines monitoring conditions and the optimal corresponding delay times based on an off-line analysis of historical alerts and incident tickets. Potential monitoring conditions are built on a set of predictive rules which are automatically generated by a rule-based learning algorithm with coverage, confidence and rule complexity criteria. These conditions and delay times are propagated as configurations into run-time monitoring systems. Moreover, a part of misconfigured monitoring conditions can be corrected according to false negative tickets that are discovered by another text classification algorithm in this framework. This paper also provides implementation details of a program product that uses this framework and shows some illustrative examples of successful results.

show abstract

“…In system management, log and system event analysis is a fundamental method to maintain, diagnose and optimize large production systems [36,37,33,31,34,32]. Log event search as a basic functionality is embedded in many system management, log analysis and system monitoring products [1,3,2].…”

Section: Related Workmentioning

confidence: 99%

Searching similar segments over textual event sequences

Tang

Chen

et al. 2013

Proceedings of the 22nd ACM International Conference on Conference on Information &Amp; Knowledge Management - CIKM '13

Self Cite

View full text Add to dashboard Cite

Sequential data is prevalent in many scientific and commercial applications such as bioinformatics, system security and networking. Similarity search has been widely studied for symbolic and time series data in which each data object is a symbol or numeric value. Textual event sequences are sequences of events, where each object is a message describing an event. For example, system logs are typical textual event sequences and each event is a textual message recording internal system operations, statuses, configuration modifications or execution errors. Similar segments of an event sequence reveal similar system behaviors in the past which are helpful for system administrators to diagnose system problems. Existing search indexing for textual data only focuses on unordered data. Substring matching methods are able to efficiently find matched segments over a sequence, however, their sequences are single values rather than texts. In this paper, we propose a method, suffix matrix, for efficiently searching similar segments over textual event sequences. It provides an integration of two disparate techniques: localitysensitive hashing and suffix arrays. This method also supports the k-dissimilar segment search. A k-dissimilar segment is a segment that has at most k dissimilar events to the query sequence. By using random sequence mask proposed in this paper, this method can have a high probability to reach all k-dissimilar segments without increasing much search cost. We conduct experiments on real system log data and the experimental results show that our proposed method outperforms alternative methods using existing techniques.

show abstract

Discovering lag intervals for temporal dependencies

Cited by 25 publications

References 28 publications

What's the big deal about big data?

What's the big deal about big data?

An integrated framework for optimizing automatic monitoring systems in large IT infrastructures

Searching similar segments over textual event sequences

Contact Info

Product

Resources

About