Prioritizing Unit Testing Effort Using Software Metrics and Machine Learning Classifiers (S)

Abstract

Unit testing plays a crucial role in object-oriented software quality assurance. Unfortunately, software testing is often conducted under severe pressure due to limited resources and tight time constraints. Therefore, testing efforts have to be focused, particularly on critical classes. As a consequence, testers do not usually cover all software classes. Prioritizing unit testing effort is a crucial task. We previously investigated a unit testing prioritization approach based on software information histories. We analyzed different attributes of ten open-source Java software systems tested using the JUnit framework. We used machine learning classifiers (Multivariate Logistic Regression and Naïve Bayes) to obtain, for each system, a set of classes to be tested. The obtained sets of candidate classes have been compared to the sets of classes for which JUnit test cases have been actually developed by testers. The cross system validation (CSV) technique results showed, among others, that the sets of candidate classes suggested by machine learning classifiers properly reflect the testers' selection. In this paper, we extend our previous work by investigating more classifiers and using leave one system out validation (LOSOV) technique. This LOSOV technique uses a combination of training datasets from different systems. The obtained results indicate that: (1) the new classifiers correctly suggest classes to be tested, and (2) tested classes are particularly well predicted in the case of large-size systems.

A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning

“…The class size [108] and an estimation of its fault-proneness [88] are less common attributes. In general, all these numerical metrics measure the complexity and level of dependency of classes, guiding the selection of those fault-prone modules that should be tested first [30,108]. Similarly, component instability, i.e.…”

“…Among the code elements from which an attribute is computed, classes constitute the finestgrained entity with 13 metrics currently used in the literature. The six metrics defined by Chidamber and Kemerer [23] have appeared in TCP studies [29,78,101,108]. The class size [108] and an estimation of its fault-proneness [88] are less common attributes.…”

“…The six metrics defined by Chidamber and Kemerer [23] have appeared in TCP studies [29,78,101,108]. The class size [108] and an estimation of its fault-proneness [88] are less common attributes. In general, all these numerical metrics measure the complexity and level of dependency of classes, guiding the selection of those fault-prone modules that should be tested first [30,108].…”

“…Classification also serves to extract knowledge from the SUT that become the input for TCP techniques. Some examples are the identification of fault-prone classes [10,70,108] and defective modules [101], the prediction of whether a change will cause test cases to fail [105], and the estimation of the execution probability of a part of the SUT [112]. • Unsupervised learning.…”

Automatic assessment of object oriented programming assignments with unit testing in Python and a real case assignment

“…About (ii), there are many works of unit testing for learning introductory programming courses (e.g., [9], [10], and [11]). Regarding (iii), among others, some authors apply ML to OOP (e.g., [12]), while other works focus on automatically generating test cases for unit testing (e.g., [13], [14] and [15]).…”