Weighted Execution Profiles for Software Testing

Farjo, Joan; Masri, Wes

doi:10.1109/icstw.2014.33

Cited by 1 publication

(1 citation statement)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The heuristic conjectures that a program element that is not covered by many tests should be considered more relevant to failure and assigned a higher score, because in typical test suites, the number of failing test cases is much smaller than the number of passing test cases . Given a test suite T and a program element pe i , the score assigned to pe i is Score(pe

_{i}) = 1 - \frac{count ((), {pe}_{i})}{| T |}

, where count ( pe i ) is the number of tests covering pe i ; and if pe i never occurred in any of the test cases, then Score ( pe i ) is 0.…”

Section: Reduction Techniquesmentioning

confidence: 99%

Reducing execution profiles: techniques and benefits

Farjo

Assi

Masri

2014

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

The interest in leveraging data mining and statistical techniques to enable dynamic program analysis has increased tremendously in recent years. Researchers have presented numerous techniques that mine and analyze execution profiles to assist software testing and other reliability enhancing approaches. Previous empirical studies have shown that the effectiveness of such techniques is likely to be impacted by the type of profiled program elements. This work further studies the impact of the characteristics of execution profiles by focusing on their size; noting that a typical profile comprises a large number of program elements, in the order of thousands or higher. Specifically, the authors devised six reduction techniques and comparatively evaluated them by measuring the following: (1) reduction rate; (2) information loss; (3) impact on two applications of dynamic program analysis, namely, cluster-based test suite minimization (App-I), and profilebased online failure and intrusion detection (App-II). The results were promising as the following: (a) the average reduction rate ranged from 92% to 98%; (b) three techniques were lossless and three were slightly lossy; (c) reducing execution profiles exhibited a major positive impact on the effectiveness and efficiency of App-I; and (d) reduction exhibited a positive impact on the efficiency of App-II, but a minor negative impact on its effectiveness. the transitivity relationships induced by control and data dependences [4], which suggests that high levels of reduction might be achieved.Hereafter, a profiled program element will also be referred to as program element for short; noting that in the context of this work, program elements represent covered statements, branches, def-uses, information flow pairs [5], slice pairs [5], paths [6], and possibly other program constructs that are also structural in nature.A well-established approach that the authors previously used [7] to reduce the high dimensionality and redundancy in execution profiles is PCA [8,9]. But PCA transforms the original data to a new coordinate system, which is problematic for some applications of dynamic program analysis because this might negatively impact the interpretability of learning models and the extraction of useful intrinsic properties.This work focuses on reduction techniques that preserve the original coordinate system to evade the limitation that PCA suffers from. In other words, the concern is with feature selection techniques [10] as opposed to feature extraction techniques [8]. Specifically, the authors will investigate different search strategies for feature selection based on the following: (1) an information theoretic approach [11], namely, the symmetric uncertainty measure (SU) (2) a heuristic randomized search approach, specifically the genetic algorithm (GA); and (3) a hybrid approach, which combines (1) and (2).The presented techniques are evaluated by measuring the following: (1) reduction rate;(2) information loss; and (3) impact on two applications of dynamic program analy...

show abstract