Exploiting the empirical characteristics of program dependences for improved forward computation of dynamic slices

ACM Trans. Softw. Eng. Methodol.

2014

Self Cite

Researchers have argued that for failure to be observed the following three conditions must be met: C R = the defect was reached; C I = the program has transitioned into an infectious state; and C P = the infection has propagated to the output. Coincidental Correctness (CC) arises when the program produces the correct output while condition C R is met but not C P . We recognize two forms of coincidental correctness, weak and strong. In weak CC, C R is met, whereas C I might or might not be met, whereas in strong CC, both C R and C I are met. In this work we first show that CC is prevalent in both of its forms and demonstrate that it is a safety reducing factor for Coverage-Based Fault Localization (CBFL). We then propose two techniques for cleansing test suites from coincidental correctness to enhance CBFL, given that the test cases have already been classified as failing or passing. We evaluated the effectiveness of our techniques by empirically quantifying their accuracy in identifying weak CC tests. The results were promising, for example, the better performing technique, using 105 test suites and statement coverage, exhibited 9% false negatives, 30% false positives, and no false negatives nor false positives in 14.3% of the test suites. Also using 73 test suites and more complex coverage, the numbers were 12%, 19%, and 15%, respectively.

Section: Coverage-based Fault Localizationmentioning

confidence: 99%

Prevalence of coincidental correctness and mitigation of its impact on fault localization

ACM Trans. Softw. Eng. Methodol.

2014

Self Cite

“…A typical profile comprises information that approximates the execution path of a program [2], specifically, the frequency of occurrence of program elements that are structural in nature, such as statements, branches, def-use pairs, slice-pairs [7] [6], and dependence chains [1]. It is plausible that some defects might not be characterized by a failing path or a set of structural program elements.…”

Section: Introduction and Related Workmentioning

confidence: 99%

State Profiling of Internal Variables

Daou

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops

2014

Self Cite

Execution profiles form the basis of many dynamic program analysis techniques developed to solve problems in fields such as software testing, fault localization, and program comprehension. A typical profile comprises information that approximates the execution path of a program, specifically, the frequency of occurrence of program elements that are structural in nature, such as statements, branches, and def-use pairs. It is plausible that some defects might not be characterized by a failing path or a set of structural program elements. In such cases, execution profiles that are based on structural elements would not be of much benefit. We argue for the need to leverage the state of the subject program, i.e., to analyze the values of representative internal variables that approximate the program state. For that purpose we: 1) built a profiling tool that captures the values taken by select variables; 2) devised three metrics to quantify the dissimilarities between our state-based execution profiles; and 3) conducted ten case studies to demonstrate the effectiveness of our tool and metrics. The majority of our case studies (8 out of 10), using all metrics, showed that failing profiles are most dissimilar from passing profiles.

“…The clustering techniques are applied onto execution profiles comprising profiling elements that varied in terms of complexity, e.g., statements, edges, def-uses, information flow pairs [17], slice pairs [11] [19], and paths [21]. Also, typically these execution profiles exhibit high dimensionality, i.e., include thousands or even hundreds of thousands of profiling elements.…”

Section: Introductionmentioning

confidence: 99%

Does Principal Component Analysis Improve Cluster-Based Analysis?

Farjo

2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops

et al. 2013

Self Cite

Researchers in the dynamic program analysis field have extensively used cluster analysis to address various problems. Typically, the clustering techniques are applied onto execution profiles having high dimensionality (i.e., involving a large number of profiling elements), sometimes in the order of thousands or even hundreds of thousands. Our concern is that the high number of profiling elements might diminish the effectiveness of the clustering process, which led us to explore the use of dimensionality reduction techniques as a preprocessing step to clustering. Specifically, in this work, we used PCA (Principal Component Analysis) as a dimensionality reduction technique and investigated its impact on two cluster-based analysis techniques, one aiming at identifying coincidentally correct tests, and the other at test suite minimization. In other words, we tried to assess whether PCA improves cluster-based analysis. Our experimental results showed that the impact was positive on the first technique, but inconclusive on the second, which calls for further investigation in the future.