Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria

Hutchins, M.; Foster, H. D.; Goradia, Tarak; Ostrand, Thomas J.

doi:10.1109/icse.1994.296778

Cited by 544 publications

(657 citation statements)

References 16 publications

Supporting

Mentioning

648

Contrasting

Unclassified

Order By: Relevance

“…This is especially important when assessing the performance of a research approach. A large body of the literature has resorted to extensive empirical studies for devising a reliable experimental protocol [30][31][32]. Recently, Allix et al have proposed a large-scale empirical studies on the dataset sizes used in the assessment of machine learning-based malware detection approaches [23].…”

Section: Related Workmentioning

confidence: 99%

Are Your Training Datasets Yet Relevant?

Allix

Bissyandé

Klein

et al. 2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

In this paper, we consider the relevance of timeline in the construction of datasets, to highlight its impact on the performance of a machine learning-based malware detection scheme. Typically, we show that simply picking a random set of known malware to train a malware detector, as it is done in many assessment scenarios from the literature, yields significantly biased results. In the process of assessing the extent of this impact through various experiments, we were also able to confirm a number of intuitive assumptions about Android malware. For instance, we discuss the existence of Android malware lineages and how they could impact the performance of malware detection in the wild.

show abstract

Section: Related Workmentioning

confidence: 99%

Are Your Training Datasets Yet Relevant?

Allix

Bissyandé

Klein

et al. 2015

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…Despite results in the literature for the Siemens subjects showing that the whole-program counterparts of BR and, especially, DU, perform better than random testing [8], these strategies were not useful for the parts of the program affected by each change. This result can be explained by the low coverage levels attained for BR and DU in these subjects, which ranged approximately between 50-90%, and which were lower than for whole-program testing for which 100% coverage was often achieved in the same subjects.…”

Section: Results and Analysismentioning

confidence: 72%

“…Therefore, we decided to start our investigation on a number of subjects from the Siemens suite [8] that we translated from C to Java. These subjects are listed in Table I, where the columns show, respectively, the name of the subject, a short description, the size in lines of code (LOC), the number of test cases available, and the number of changes used in our study.…”

Section: ) Implementationmentioning

confidence: 99%

Applying aggressive propagation-based strategies for testing changes

Santelices

Harrold

2011

2011 Fourth IEEE International Conference on Software Testing, Verification and Validation

View full text Add to dashboard Cite

Abstract-Test-suite augmentation for evolving softwarethe process of augmenting a test suite to adequately test software changes-is necessary for any program that undergoes modifications as part of its development and maintenance cycles. Recently, we presented a new technique for test-suite augmentation based on leveraging the propagation conditions for the effects of changes. Although empirical studies show that this technique can be quite effective for testing changes, the experiments have been limited because of the complexity of the implementation. In this paper, we present a new and more efficient approach for propagation-based testing of changes that can reach much longer propagation-distances and can focus the testing more precisely on those behaviors of changes that can actually affect the output. Using an implementation of this new approach, we performed a study on a set of changes on Java programs for which we compared, to a much larger extent, our propagation-based strategy with other existing techniques for testing changes. The results of the study not only confirm the superior effectiveness of propagation-based strategies over these other techniques for testing changes, but also quantify that superiority and clarify the conditions under which our approach is most effective.

show abstract

“…To evaluate bug localisation techniques, the Siemens Programs [15] are often used [1,2,13] as a reference suite of C programs artificially instrumented with different bugs. More specifically, it usually is just a small subset of this benchmark which is used.…”

Section: Discussionmentioning

confidence: 99%

“…If the precision rises significantly when adding graphs containing a certain method, this method is deemed more likely to contain a bug. Experiments with five out of 130 bugs from the Siemens Programs [15] demonstrate good classification performance, but do not evaluate the precision of the bug localisation. Furthermore, the authors do not generate a ranking of methods suspected to contain a bug.…”

Section: Call Graph Based Fault Detectionmentioning

confidence: 99%

Mining Edge-Weighted Call Graphs to Localise Software Bugs

Eichinger¹,

Böhm²,

Huber³

Machine Learning and Knowledge Discovery in Databases

View full text Add to dashboard Cite

An important problem in software engineering is the automated discovery of noncrashing occasional bugs. In this work we address this problem and show that mining of weighted call graphs of program executions is a promising technique. We mine weighted graphs with a combination of structural and numerical techniques. More specifically, we propose a novel reduction technique for call graphs which introduces edge weights. Then we present an analysis technique for such weighted call graphs based on graph mining and on traditional feature selection schemes. The technique generalises previous graph mining approaches as it allows for an analysis of weights. Our evaluation shows that our approach finds bugs which previous approaches cannot detect so far. Our technique also doubles the precision of finding bugs which existing techniques can already localise in principle.

show abstract

Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria

Abstract: This paper reports an experimental study investigating

Cited by 544 publications

References 16 publications

Are Your Training Datasets Yet Relevant?

Are Your Training Datasets Yet Relevant?

Applying aggressive propagation-based strategies for testing changes

Mining Edge-Weighted Call Graphs to Localise Software Bugs

Contact Info

Product

Resources

About