Explore the effect of size stratification of datasets on software faults proneness prediction performance

Abstract

Size is one of the significant factors associated with bugs, and it has been used to predict software faults. We believe that stratifying software files based on size can play an essential role in improving prediction performance. This study explores the effect of size by stratifying our sample based on each unit’s size and distributing software units in multiple stratified groups based on an equal distribution approach. We stratified the Eclipse Europa project and Derby project files, and we reported the performance of each stratified group and compared them. We used two popular classifiers, decision tree J48 and random forest, to implement this experiment. These classifiers presented similar results on the same group of files. The results indicated that predicting faults with large files is significantly better than predicting those with small files. Furthermore, the files of large groups provide reliable and more stable performance. Finally, we resampled all the stratified groups of the two projects to explore if this can impact the performance. We found that the resampling has no impact on the performance of algorithms using the Europa dataset. However, the effect was significant when the Derby project was used.