An Efficient Format for Nearly Constant-Time Access to Arbitrary Time Intervals in Large Trace Files

Chan, Anthony; Gropp, William; Lusk, Ewing L.

doi:10.1155/2008/749874

Cited by 41 publications

(39 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Hence, to store the interval values, any interval container, such as an R-tree [16], the SLOG2 file format [17] or a State History Tree [18] can be used.…”

Section: History Data Storementioning

confidence: 99%

Cube data model for multilevel statistics computation of live execution traces

Ezzati‐Jivan

Dagenais

2014

Concurrency and Computation

View full text Add to dashboard Cite

Execution trace logs are used to analyze system run-time behaviour and detect problems. Trace analysis tools usually read the input logs and gather either a detailed or brief summary of them to later process and inspect in the analysis steps. However, continuous and lengthy trace streams contained in the live tracing mode make it difficult to indefinitely record all events or even a detailed summary of the whole stream. This situation is further complicated when the system aims to compare different parts of the trace and provide a multilevel and multidimensional analysis.This paper presents an architecture with corresponding data structures and algorithms to process stream events, generate an adequate summary -detailed enough for recent data and succinct enough for old data-and organize them to enable an efficient multilevel and multidimensional analysis, similar to OLAP analyses in the database applications. The proposed solution arranges data in a compact manner using interval forms and enables the range queries for any arbitrary time durations. Since this feature makes it possible to compare of different system parameters in different time areas it significantly influences the systems ability to provide a comprehensive trace analysis. Although the Linux operating system trace logs are used to evaluate the solution, we propose a generic architecture which can be used to summarize various types of stream data.

show abstract

“…Hence, to store the interval values, any interval container, such as an R-tree [16], the SLOG2 file format [17] or a State History Tree [18] can be used.…”

Section: History Data Storementioning

confidence: 99%

Cube data model for multilevel statistics computation of live execution traces

Ezzati‐Jivan

Dagenais

2014

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…One of the most frequently used visualizations is that of traces of MPI messages sent. This is used in tools such as ParaGraph [14], JumpShot [8], and Vampir [17]. Conceptually, these views are similar to Gantt charts, and consequently rarely scale well, so Muelder et al use visualizations of MPI function call durations to improve the visualizations of MPI traces [23].…”

Section: Communication Analysis To Improve Performancementioning

confidence: 99%

Visualizing Network Traffic to Understand the Performance of Massively Parallel Simulations

Landge

Levine

Bhatelé

et al. 2012

IEEE Trans. Visual. Comput. Graphics

View full text Add to dashboard Cite

Fig. 1. Network traffic resulting from two different runs of the parallel simulation pF3D. This simulation models laser plasma interaction inside of a hohlraum chamber by decomposing the domain into a set of blocks (left). Depending on how data blocks are mapped to processor cores (middle), different communication patterns occur. When staggering data placement (bottom right) we observe significantly more balanced communication compared to a default mapping similar to how the domain is decomposed (top right).Abstract-The performance of massively parallel applications is often heavily impacted by the cost of communication among compute nodes. However, determining how to best use the network is a formidable task, made challenging by the ever increasing size and complexity of modern supercomputers. This paper applies visualization techniques to aid parallel application developers in understanding the network activity by enabling a detailed exploration of the flow of packets through the hardware interconnect. In order to visualize this large and complex data, we employ two linked views of the hardware network. The first is a 2D view, that represents the network structure as one of several simplified planar projections. This view is designed to allow a user to easily identify trends and patterns in the network traffic. The second is a 3D view that augments the 2D view by preserving the physical network topology and providing a context that is familiar to the application developers. Using the massively parallel multi-physics code pF3D as a case study, we demonstrate that our tool provides valuable insight that we use to explain and optimize pF3D's performance on an IBM Blue Gene/P system.

show abstract

“…For parallel applications, tools such as Vampir [19], Jumpshot [20] or the Intel Trace Analyzer and Collector [21] record all message transfers between MPI processes and store them to disk with complete timestamps. They can then display the collected trace on a timeline.…”

Section: B Parameter Redistribution Bottleneckmentioning

confidence: 99%

Reconciling Sampling and Direct Instrumentation for Unintrusive Call-Path Profiling of MPI Programs

Szebenyi

Gamblin

Schulz

et al. 2011

2011 IEEE International Parallel &Amp; Distributed Processing Symposium

View full text Add to dashboard Cite

Abstract-We can profile the performance behavior of parallel programs at the level of individual call paths through sampling or direct instrumentation. While we can easily control measurement dilation by adjusting the sampling frequency, the statistical nature of sampling and the difficulty of accessing the parameters of sampled events make it unsuitable for obtaining certain communication metrics, such as the size of message payloads. Alternatively, direct instrumentation, which is preferable for capturing message-passing events, can excessively dilate measurements, particularly for C++ programs, which often have many short but frequently called class member functions. Thus, we combine these techniques in a unified framework that exploits the strengths of each approach while avoiding their weaknesses: We use direct instrumentation to intercept MPI routines while we record the execution of the remaining code through low-overhead sampling. One of the main technical hurdles mastered was the inexpensive and portable determination of call-path information during the invocation of MPI routines. We show that the overhead of our implementation is sufficiently low to support substantial performance improvement of a C++ fluid-dynamics code.

show abstract

An Efficient Format for Nearly Constant-Time Access to Arbitrary Time Intervals in Large Trace Files

Cited by 41 publications

References 13 publications

Cube data model for multilevel statistics computation of live execution traces

Cube data model for multilevel statistics computation of live execution traces

Visualizing Network Traffic to Understand the Performance of Massively Parallel Simulations

Reconciling Sampling and Direct Instrumentation for Unintrusive Call-Path Profiling of MPI Programs

Contact Info

Product

Resources

About