Fine grained kernel logging with KLogger

Etsion, Yoav; Tsafrir, Dan; Kirkpatrick, Scott; Feitelson, Dror G.

doi:10.1145/1272996.1273023

Cited by 11 publications

(4 citation statements)

References 18 publications

(5 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The harder part was measuring the duration of the handler. For this purpose we have instrumented the kernel by using klogger, a kernel logger we developed that supports efficient fine-grain events [20]. While the code is integrated into the kernel, its activation at runtime is controlled by applying a special sysctl call using the /proc file system.…”

Section: Direct Overheadmentioning

confidence: 99%

The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops)

Tsafrir

2007

Proceedings of the 2007 Workshop on Experimental Computer Science

Self Cite

View full text Add to dashboard Cite

The overhead of a context switch is typically associated with multitasking, where several applications share a processor. But even if only one runnable application is present in the system and supposedly runs alone, it is still repeatedly preempted in favor of a different thread of execution, namely, the operating system that services periodic clock interrupts. We employ two complementing methodologies to measure the overhead incurred by such events and obtain contradictory results.The first methodology systematically changes the interrupt frequency and measures by how much this prolongs the duration of a program that sorts an array. The overall overhead is found to be 0.5-1.5% at 1000 Hz, linearly proportional to the tick rate, and steadily declining as the speed of processors increases. If the kernel is configured such that each tick is slowed down by an access to an external time source, then the direct overhead dominates. Otherwise, the relative weight of the indirect portion is steadily growing with processors' speed, accounting for up to 85% of the total.The second methodology repeatedly executes a simplistic loop (calibrated to take 1ms), measures the actual execution time, and analyzes the perturbations. Some loop implementations yield results similar to the above, but others indicate that the overhead is actually an order of magnitude bigger, or worse. The phenomenon was observed on IA32, IA64, and Power processors, the latter being part of the ASC Purple supercomputer. Importantly, the effect is greatly amplified for parallel jobs, where one late thread holds up all its peers, causing a slowdown that is dominated by the per-node latency (numerator) and the job granularity (denominator). We trace the bizarre effect to an unexplained interrupt/loop interaction; the question of whether this hardware misfeature is experienced by real applications remains open.

show abstract

Section: Direct Overheadmentioning

confidence: 99%

The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops)

Tsafrir

2007

Proceedings of the 2007 Workshop on Experimental Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…This total time is accumulated in numerous short bursts of CPU activity, interspersed by system activity (handling interrupts or system calls) and possibly the execution of other processes. Obtaining data about the length of CPU bursts requires precise instrumentation, using tools such as KLogger [219]. Using such tools, it is found that typical bursts are very short, and in particular they are much shorter than the operating system's time quantum (Figure 9.6) -so much so as to make the operating system quantum irrelevant.…”

Section: Process Runtimesmentioning

confidence: 99%

Workload Modeling for Computer Systems Performance Evaluation

Feitelson

2015

Self Cite

191

131

View full text Add to dashboard Cite

Reliable performance evaluations require the use of representative workloads. This is no easy task since modern computer systems and their workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.

show abstract

“…The actual source code of the Linux kernel is modified to include LTT macros in specified functions and LTT based data structures for holding the trace information. Other tools that fall into this category are K42 [14] and KLogger [18]. All of these only provide tracing.…”

Section: Related Workmentioning

confidence: 99%

Early Experiences with KTAU on the IBM BG/L

Nataraj

Malony

Morris

et al. 2006

Euro-Par 2006 Parallel Processing

View full text Add to dashboard Cite

The influences of OS and system-specific effects on application performance are increasingly important in high performance computing. In this regard, OS kernel measurement is necessary to understand the interrelationship of system and application behavior. This can be viewed from two perspectives: kernel-wide and process-centric. An integrated methodology and framework to observe both views in HPC systems using OS kernel measurement has remained elusive. We demonstrate a new tool called KTAU (Kernel TAU) that aims to provide parallel kernel performance measurement from both perspectives. KTAU extends the TAU performance system with kernel-level monitoring, while leveraging TAU's measurement and analysis capabilities. As part of the ZeptoOS scalable operating systems project, we report early experiences using KTAU in ZeptoOS on the IBM BG/L system.

show abstract

Fine grained kernel logging with KLogger

Cited by 11 publications

References 18 publications

The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops)

The context-switch overhead inflicted by hardware interrupts (and the enigma of do-nothing loops)

Workload Modeling for Computer Systems Performance Evaluation

Early Experiences with KTAU on the IBM BG/L

Contact Info

Product

Resources

About