Toddler: Detecting performance problems via similar memory-access patterns

Nistor, Adrian; Song, Linhai; Marinov, Darko; Lu, Shan

doi:10.1109/icse.2013.6606602

Cited by 96 publications

(100 citation statements)

References 30 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each of these tools detects one specific type of performance bugs, such as inefficient nested loops [37], under-utilized data structures [46], and temporary object bloat [12,44,45], through static or dynamic program analysis. They are not designed to cover a wide variety of performance bugs.…”

Section: Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Statistical debugging for real-world performance problems

Song

2014

Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages &Amp; Applications

Self Cite

View full text Add to dashboard Cite

Design and implementation defects that lead to inefficient computation widely exist in software. These defects are difficult to avoid and discover. They lead to severe performance degradation and energy waste during production runs, and are becoming increasingly critical with the meager increase of single-core hardware performance and the increasing concerns about energy constraints. Effective tools that diagnose performance problems and point out the inefficiency root cause are sorely needed.The state of the art of performance diagnosis is preliminary. Profiling can identify the functions that consume the most computation resources, but can neither identify the ones that waste the most resources nor explain why. Performance-bug detectors can identify specific type of inefficient computation, but are not suited for diagnosing general performance problems. Effective failure diagnosis techniques, such as statistical debugging, have been proposed for functional bugs. However, whether they work for performance problems is still an open question.In this paper, we first conduct an empirical study to understand how performance problems are observed and reported by real-world users. Our study shows that statistical debugging is a natural fit for diagnosing performance problems, which are often observed through comparison-based approaches and reported together with both good and bad inputs. We then thoroughly investigate different design points in statistical debugging, including three different predicates and two different types of statistical models, to understand which design point works the best for performance diagnosis. Finally, we study how some unique nature of performance bugs allows sampling techniques to lower the over- * Shan is now with University of Chicago.

show abstract

Section: Motivationmentioning

confidence: 99%

“…Liu and Berger [31] build two tools to attack the false sharing problem in multithreaded software. There are also tools that detect inefficient nested loops [37] and workload-dependent loops [43].…”

Section: Performance Bug Detectionmentioning

confidence: 99%

Statistical debugging for real-world performance problems

Song

2014

Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages &Amp; Applications

Self Cite

View full text Add to dashboard Cite

show abstract

“…Previous study has shown that this type of performance-related software defects 1 widely exist in real- 1 We will refer to these defects as performance bugs or performance problems interchangeably following previous work in this area [18,21,30]. world [9,18,21,28,34].…”

Section: Motivationmentioning

confidence: 99%

“…Each of these tools detects one specific type of performance bugs, such as inefficient nested loops [30], under-utilized data structures [38], and temporary object bloat [11,36,37], through static or dynamic program analysis. They are not designed to cover a wide variety of performance bugs.…”

Section: Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Statistical debugging for real-world performance problems

SongLinhai

2014

SIGPLAN Not.

Self Cite

View full text Add to dashboard Cite

Design and implementation defects that lead to inefficient computation widely exist in software. These defects are difficult to avoid and discover. They lead to severe performance degradation and energy waste during production runs, and are becoming increasingly critical with the meager increase of single-core hardware performance and the increasing concerns about energy constraints. Effective tools that diagnose performance problems and point out the inefficiency root cause is sorely needed.The state of the art of performance diagnosis is preliminary. Profiling can tell where computation resources are spent, but not where and why the resources are wasted. Performance-bug detectors can identify specific type of inefficient computation, but are not suited for diagnosing general performance problems. Effective failure diagnosis techniques, such as statistical debugging, have been proposed for functional bugs. However, whether they work for performance problems is still an open question.In this paper, we first conduct an empirical study to understand how performance problems are observed and reported by real-world users. Our study shows that statistical debugging is a natural fit for diagnosing performance problems, which are often observed through comparison-based approaches and reported together with both good and bad inputs. We then thoroughly investigate different design points in statistical debugging, including three different predicates and two different types of statistical models, to understand which design point works the best for performance diagnosis. Finally, we study how some unique nature of performance bugs allows sampling techniques to lower the overhead of run-time performance diagnosis without extending the diagnosis latency.

show abstract

A systematic mapping study of software performance research

Han

Yan

2023

Softw Pract Exp

View full text Add to dashboard Cite

Summary Context: Software performance is crucial for ensuring the quality of software products. As one of the non‐functional requirements, the few efforts devoted to software performance have often been neglected until a later phase in the software development life cycle (SDLC). The lack of clarity of what software performance research literature is available prevents researchers from understanding what software performance research fields are available. It also creates difficulty for practitioners to adopt state‐of‐the‐art software performance techniques. Software performance research is not as organized as other established research topics such as software testing. Thus, it is essential to conduct a systematic mapping study as a first step to provide an overview of the latest research literature available in software performance. Objective: The objective of this systematic mapping study is to survey and map software performance research literature into suitable categories and to synthesize the literature data for future access and reference. Method: This systematic mapping study conducts a manual examination by querying research literature in noble journals and proceedings in software engineering in the past decade. We examine each paper manually and identify primary studies for further analysis and synthesis according to the pre‐defined inclusion criteria. Lastly, we map the primary studies based on their corresponding classification category. Results: This systematic mapping study provides a state‐of‐the‐art literature mapping in software performance research. We have carefully examined 222 primary studies out of 2000+ research literature. We have identified six software performance research categories and 15 subcategories. We generate the primary study mapping and report five research findings. Conclusions: Unlike established research fields, it is unclear what types of software performance research categories are available to the community. This work takes the systematic mapping study approach to survey and map the latest software performance research literature. The study results provide an overview of the paper distribution and a reference for researchers to navigate research literature on software performance.

show abstract

Toddler: Detecting performance problems via similar memory-access patterns

Cited by 96 publications

References 30 publications

Statistical debugging for real-world performance problems

Statistical debugging for real-world performance problems

Statistical debugging for real-world performance problems

A systematic mapping study of software performance research

Contact Info

Product

Resources

About